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PART I: SUMMARY 



The general purpose of the various activities included in the' 
Oregon Planning, Programming, Budgeting Systems Project was to 
enhance the quality of planning, program development, financial 
arrangements, and evaulation efforts in the vocational education 
establishment at state and local levels. 

Three general areas of activity were proposed and accomplished 
in the Project. The major activity was that of conducting a two- 
week Institute in Planning, Programming, Budgeting Systems (PPBS) 
for participants from state departments and state divisions of 
vocational education. The planning and recruitment of participants * 
was accomplished during the months immediately prior to the 
Institute, which was held August 19 - 30, 1968. The major portion of 
this final report is directly appropriate to that Institute. Part III 
includes a detailed, descriptive report, which not only is a report 
of an institute, but includes major instructional, content material 
on Planning, Programming, Budgeting Systems. Much bibliographical 
material was collected and utilized during the Institute and is made 
available ih the various appendices as well as in Part III. 

A second major activity of the Project was conferring with 
appropriate representatives from state level, vocational education 
agencies, who had had varying degrees of experience with PPBS. 

Particular emphasis in these conferences was the focus on problems 
and needs with implications for programmatic content for the Oregon 
PPBS Institute. 

An early and continuing liaison with The Center for Research and 
Development in Vocational-Technical Education, The Ohio State University 
was established. The Ohio Center held a similar grant which financed 
the Oregon PPBS Institute and conducted their Institute in October, 

1968. This liaison resulted in cooperative endeavors with respect to 
consultation on program content, materials, bibliographies, case studies 
methodology, evaluation, and recruitment. Post-Institute cooperation 
continues with emphasis on refinement of PPBS curriculum materials and 
planning for appropriate followup. 

A "third party" evaluation was conducted for the Oregon PPB£ 
Institute and is included in this report as Part IV. 

Four major recommendations arise from Teaching Research staff’s 
experience with the Planning, Programming, Budgeting Systems Project: 

1) A research effort is needed if the potential of the PPB 
systems is to be realized. It is recommended that effort be focused on 
cost benefit and cost effectiveness analysis. Of particular need, is 
the development of cost effectiveness models which can serve as a basis 



for comparative cost studies in vocational education. The goal would 
be to establish criteria for funding future vocational education pro- 
grams. Major attention is also needed in researching approaches and 
establishing models for determining the dollar value of program bene- 
fits. A successful effort is necessary if cost benefit ratios, for 
vocational programs are to contribute meaningfully to decision making. 
Work should also begin on evaluating the effectiveness of PPBS con- 
cepts as applied to education. 

2) A major development effort is needed and recommended. This 
effort should include adaptation and application of guides and models 
for agencies and institutions of various levels and types. An over- 
all theoretical base and guide to application to the educational 
enterprise is needed as well as materials on specific application. 

These could also serve as basic instructional materials for training. 

3) A comprehensive training effort should be implemented with 
a view to the degree of dissemination necessary to insure that PPBS 
has a fair trial. Special training efforts for advanced PPBS students 
and analysts should be included. 

4) A pilot project for four to six states should be inaugurated 
to include most of the foregoing and to establish a volume of research 
data and practical experience necessary for sensible development of the 
Planning, Programming, Budgeting Systems. 



- 2 - 



PART II: 



INTRODUCTION TO THE PLANNING, 
PROGRAMMING, AND BUDGETING 
SYSTEMS INSTITUTE. 



The cost of education in the United States is currently about 
$38 billion dollars annually. However, there is substantial agree- 
ment that adequate monies are not currently made available to meet 
the needs. There are recurring voices from the private sector with 
direct reference to administering educational funds in a more business- 
like manner. The concept of educational funds being an investment in 
human endeavor has been widely accepted. As yet, however, little 
progress has been made in management of the educational establish- 
ment based on basic economic principles . 

One of the promising developments iii this aspect of education, 
and particularly in vocational education, is that of Planning, Pro- 
gramming, and Budgeting Systems. The purpose of PPBS, based on prin- 
ciples of classical economics, is to provide a method of planning, 
management, and control in order to realize the greatest possible 
gain from the investment of capital in the educational enterprise. 

Those involved in making money decisions for education are 
becoming increasingly critical and resistive in allocating larger 
and larger sums on traditional (incremental) bases of budget requests. 
These persons are looking increasingly for evidence that available 
resources are spent in the most effective and economical manner pos- 
sible. PPBS furnishes a vehicle for accomplishing that purpose. 

The Research and Policy Committee of the Committee for Economic 
Development called for application of PPBS in the federal establishment 
in Budgeting for National Objectives in 1966. This statement was 
followed by Innovation in Education : New Directions for the American 

School, July, 1968, issued by the same group and issued the challenge 
to apply data systems, cost-benefit analysis, and other fundamental 
economic concepts to educational problem-solving. The American Council 
on Education made an excellent beginning in publishing "Planning for 
Effective Resource Allocation in Universities.” Several states, includ- 
ing California, Wisconsin, Oregon, Hawaii, Washington, Minnesota, and 
New York, are involved in the implementation of PPB concepts. The fed- 
eral government began to implement PPB concepts In the Department of 
Defense under the aegis of Secretary McNamara. The President initiated 
the system as a basic executive policy of the federal government in 
1965. The U. S. Office of Education has made some progress in institut- 
ing PPBS, particularly the Bureau of Vocational Education, which has 
financed three national Institutes on this topic. One of these was 
conducted under the auspices of Teaching Research Division, Oregon 
State System of Higher Education, in August, 1968. 
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Educational administrators, as v/ell as others, need assistance in 
formulating, adapting, and implementing these innovative, and hopefully, 
more effective methods of decision making. 



The Genius cf the PPBS . 

The major strengths of Planning, Programming, Budgeting are inherent 
in the system. PPB is a systems approach to educational budgeting, 
management, and control. 

Perhaps the greatest contribution which PPBS can make to educational 
organizations is that of requiring entire institutions to become goal 
centered through the specification of objectives. The efficacy of goal 
centered behavior in the motivation of human endeavor needs no 
elucidation here. Suffice it to state that the implementation of the 
systems approach should result in an organization 1 s becoming goal oriented 
at every level. 

Another major aspect of the system is in planning. PPB systems call 
for planning based on explicated objectives projected for several years 
in advance. This enables the decision-maker to see exactly what the 
proposed program is attempting to accomplish and establishes the long 
range as well as initial costs. This helps in preventing "foot-in-the- 
door” financing with the accompanying possibility of investment folly and 
long term funding squeeze. The explication of objectives will sharpen 
the entire operation and lays the foundation for evaluation of programs. 

The consideration of alternative programs to attain the same 
objectives has much to offer through stimulation of innovative program 
development. The generation of alternatives can lead to experimentation 
which could result in more economical and/or more effective programs as 
well as stimulating irxnovation aid involvement of staff in decision 
making . 

The application of systematic analyses is the heart of the PPB system. 
Although cost effectiveness and cost benefit analyses are most often 
mentioned with respect to PPBS, there are four or five other levels of 
analyses which promise much improvement over hunch, nostrum, and the 
personal influence system. Higher level analyses demand the development 
of data streams. These appear to be available in only a few sophisticated 
institutions. Cost effectiveness can answer the question, "What is the 
cost per product (output)?” Cost benefit analysis can answer the question 
in economic terms, "Is the benefit worth the cost?” These analytic 
methods can also furnish answers to comparative costs between the several 
curricula, between schools in a system, and between identical subjects in 
different institutions. The application of the economic principle of 
marginal costs shows much potential in determining the optimum effective 
economic unit in size of programs and institutions. Much research is 
needed in developing various levels of analyses. 






To have dollar decisions made on the basis of program has been 
the heretofore fruitless dream of many program-centered administrators. 
PPBS furnishes the vehicle to accomplish this goal. The program mem- 
oranda and issues sections are essential aspects of the system. Cer-- 
tainly, it seems reasonable to envision that judicious application of 
the PPBS will result in more thorough and acceptable justification of 
programs and result in higher levels of funding, even though this is 
not the primary purpose of the system. 

The expenditure of monies for any purpose is an economic phenomena. 

It would seem logical that expenditures in education, as in other sectors, 
should be based on application of sound economic principles as is en- 
visioned in application of PPBS. The basic principles of PPBS are der- 
ived from classical, capitalistic economic theory. In addition, the PPB 
vehicle should perpetuate a sense of financial awareness in the ed- 
ucational establishment with accompanying potential foe economy. The 
lay decision maker will, hopefully, be favorably impressed and assume 
a more open minded minded posture toward the fund seeker. 

P urpose of the Project . The general purpose of this project was 
to stimulate development of PPBS in the vocational educational estab- 
lishment and thus enhance the quality and effectiveness of decision 
making at state and local levels.- The supporting objectives were as 
follows: 

1) To conduct a two-week national PPBS Institute for selected 
state and local vocational education administrators. 

< 

2) To stress the need for systematic program planning to meet 
changing requirements. 

3) To provide opportunity for establishing broad professional 
contacts. 

4) To promote the partnership concept in vocational education with 
respect to federal, state, and local relationships. 

5) To acquaint participants with resources currently available 
for developing and implementing PPBS concepts. 

6) To develop and collect PPBS materials. 

7) To cooperate with a similar project at the Center for Research 
and Development of Leadership in Vocational-Technical Education, 
The Ohio State University. 

The more detailed objectives of the Oregon Planning, Programming, 
and Budgeting Systems Institute were stated in the official Institute 
Program as : 

1) To acquaint participants with educational and financial problems. 
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2) To develop need for a systematic, information data system as 
a basis for improved decision making. 

3) To thoroughly orient participants in the structure and theory 
of Planning, Programming, Budgeting Systems. 

4) To define data streams in vocational education necessary for 
use in the PPB system. 

5) To introduce methods of analyses utilized in the PPB system. 

6) To develop a guide or model for implementation purposes in 
state divisions of vocational technical education. 

I) To define problems of implementation and solutions ad developed 
by practitioners in the field. 

8) To define areas needing further research and development in 
application of PPBS to vocational education. 

9) To orient participants to the policical processes involved in 
budgetary processes. 



Description of Activitie s 

The Institute . 

The major activity of the Project was the planning, preparation, 
and conduct of a two-week Institute held August 19-30, 1968. Tne 
Institute site was on campus of the Oregon College of Education, Monmouth, 
Oregon, which also houses the home offices of the Contractor. A copy 
of the program may be found in Appendix A. It also lists the various 
consultants, speakers, and instructors. 

In selection of Institute faculty, the main purpose was that of 
balance. Those included were noted researchers in educational administra- 
tion, practicing vocational educational administrators representing sev- 
eral levels of responsibility, PPBS experts and analysts, economists, 
politicians, and state and federal bureau of budget personnel. 

The program content was geared for beginners in the PPBS field and 
was planned as a judicious mix of practical application, PPBS and 
economic theory. The need for PPBS in the context of seeking solutions 
to problems in administration of vocational education was stressed in 
opening sessions. Closing sessions concentrated on necessary planning 
for implementation when participants returned to their home situations. 

Participants were recruited on a team basis in cooperation with the 
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Ohio State Center. A team consisted of the state director for vocational- 
technical education, the assistant for budget, the program director or 
planner, and a representative of the state budget office. A selected 
few community college representatives were also included ^ The program 
was planned to present a thorough executive orientation to state directors 
and/or team leaders in the first three days. By agreement, the Oregon 
Institute included participants from states west of the Mississippi 
River and the Ohio Institute concentrated on participants east of the 
Mississippi River. Arrangements were made for some who had schedule 
conflicts to attend regardless of state of residency. A list of partic- 
ipants is shown as Appendix 3. 

A variety of methods were utilized in the Institute program. These 
included a judicious mixture of formal presentations, question and 
answer periods, discussions, small group work sessions, and individual 
reading and study. The overall program was moderately strenuous as 
viewed by the participants. 

Pre-Institute Conferences . One of the activities proposed was that 
of conferring with two State Divisions of Vocational Education having an 
interest in implementing PPBS. The purpose of these conferences was to 
pinpoint problems which they were encountering in implementing PPBS and 
their implication for the Institute program. This was an essential 
procedure in making the Institute fit the needs of participants and as 
practical as possible. 

Conferences were held with personnel of SDVE in Washington, Wis- 
consin, and Oregon. These states were selected as representative of 
three distinct levels of involvement in PPB. Wisconsin was in its 
third cycle of biennial PPBS budget preparation and had considerable 
experiences with the system. Conferences were held with Clarence 
Greiber, Director, and R. F. Birdnar, Administrative Officer, Department 
of Vocational-Technical and- Adult Education, Madision, Wisconsin. A 
discussion was also held with Dr. Robert Williams, of the Coordinating 
Council for Higher Education in Wisconsin. 

Conferences were held with Mr. Ernest Cramer, Director, and Mr. 
Richard Moe, Assistant Director, Department of Vocational-Technical 
Education, for the State of Washington. The Central Budget Agency 
had recently issued directions to submit the 1969—71 biennial proposals 
in a PPBS system and they were well into the problems encountered in 
initial implementation. 

Conferences were also held with SDVE personnel in Oregon. These 
persons were primarily at the interest level, but had a mandate from the 
state budget director to comply with a PPB approach if possible. These 
conferences were with Dr. Wm. Loomis, Assistant Superintendent of Public 
Instruction for Community Colleges and Vocational Education, and Mr. 

Albion Ringo, Director, Vocational-Technical Education for Oregon. 




These several conferences proved to be helpful and had a substantial, 
positive effect on the Institute Program. These effects were both 
substantive and procedural. A number of potential Institute instructors 
were also identified in these conferences. 

Ohio State Center Liaison . Close communication war; established 
early in the planning stages of the Project with Dr. Robert Taylor, 
Director of the Center of Research and Leadership Development in 
Vocational-Technical Education, The Ohio State University, and Dr. J. H. 
McGivney, Director of the Ohio PPBS Institute. Various aspects of joint 
planning included initial announcement and recruiting of participants 
for both Institutes, establishment of Institute dates, program planning, 
utilization of curriculum materials, potential instructional personnel, 
and various bibliography. The Director of the Oregon Institute served 
as a consultant for the second half of the Ohio Institute. The various 
critiques and evaluation of the Oregon PPBS Institute were shared with 
Ohio Institute personnel so that they might incorporate findings as 
seemed feasible in final planning of the Ohio Insitiute Program. 

Cooperative activities continue with the Oregon PPBS Institute 
Director serving on a committee at the Ohio State Center to plan and 
conduct appropriate followup activities . The activities include some 
refinement and editing of PPBS materials and planning an advanced work- 
shop for trainees who have had the basic two-week institute. 

Collection of PPBS Curriculum Materials . The collection and 
preparation of materials has resulted in an extensive bibliography 
which may be seen as Appendix C. Much of this material has been collected 
and screened for suitability in training efforts. These include case 
studies suitable for teaching purposes. Reading and study assignments 
were made from this list and materials secured in quantity for distri- 
bution and use of each participant. See Appendix D for details of Ins- 
titute reading assignments. 

Staff Orientation to PPBS . • The Director of the Oregon PPBS at- 
tended a three-week seminar on the subject in May. That seminar was 
jointly sponsored by the University of Virginia and the Bureau of Train- 
ing, Federal Civil Service Commission. A variety of publications were 
collected in preparation for the Oregon Institute and were made avail- 
able to all local staff. A full scale executive orientation was made 
to the Directorate, Teaching Research Division, and later to the entire 
professional staff of the Division. Local staff attended most sessions 
of the Institute and the Institute Director and Associate Director served 
as instructor and consultants for various topics. 

Definition of further tasks in Planning , Programming , and Budgeting 
Systems . Most succinctly stated there are three basic needs in developing 
PPBS for education. These are: (I) basic research in cost effective- 

ness and cost benefit analysis; (2) preparation of additional curriculum 
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materials; and (3) additional training efforts. 

Federal, state, and local relationships were developed through 
planning and fc^titute activities. Persons from the various levels came 
to know each other, their interests and problems in PPBS. *® rtic **? nts 
from the several states established communication relationships with a 
view to future exchange of information on developments in PPB systems. 

A variety of resources on PPBS, both personnel and written, were 
identified and made known to participants for future use. Details may 
be found in the bibliographical sources and the program consultants 
and instructors. 

The limited amount of time available for development of original 
models and guides was mostly spent on search and adaptation of earlier 
models already developed. The major original contribution developed 
was a criteria for feasibility studies of occupational programs. An 
outline of this criteria may be examined in Appendix E. A full copy 
may be secured from Teaching Research, Monmouth, Oregon. The Ohio 
State Center produced a case problem which was utilized during the 
Oregon Institute (see. Appendix F). A number of existing case studies 
were utilized for instructional purposes through the Institute. A list 
of these may be found in Appendix C. 



PART III: 



THE OREGON INSTITUTE ON PLANNING 
PROGRAMMING AND BUDGETING SYSTEM 



Introduction; The purpose of this section is to summarize and 
paraphrase the essentials of the various presentations^ and other 
Institute activities. These will follow in sequence as in the Institute 
Program itself. Other pertinent introductory material was given in 
Part II and a copy of the program appears as Appendix A. 

First and second days . The first two days were devoted to assessing 
the need for more effective decision making procedures, PPBS or other, 
in the context of state level management and financial problems in 
vocational education, an overview of the PPBS structure, and the 
definition of various tasks to be accomplished in utilizing PPBS in the 
field. These served as an executive orientation to state directors/ 
team leaders who could be present only during the opening days. 

Dr. Keith Goldhammer, Dean, College of Education, Oregon State 
University, gave the first major address, "Research in Education Decision 
Making." Dean Goldhammer introduced his remarks with reference to the 
rapidly increasing complexity of today’s shrinking world. He referred 
to the cultural lag between society and the classroom, differences in 
perceptions and reality, and the descrepancies between what we know and 
what we do. He gave extensive examples of institutional disfunctioning 
with respect to administrative practices. Thusly, the major point was 
made for the need to use knowledge and data to improve professional 
administrative practice. Dean Goldhammer* s second major point was. to 
define the educational administrator’s task with respect to the initial 
point. He stated that the administrator must be the goal setter, the 
data collector, interpreter, and relater to decision making. The 
relevant data needed in these tasks include numbers about people, society, 
and the educative process, which bear on the defined problems and goals. 

Thirdly, Dr. Goldhammer pointed out the five major functions of 
research with respect to decision making. These included: 

1) definition and collection of adequate data 

2) order the data and estimate validity 

3) store and retreival system 

4) define critical utilization of data 

5) apply theory and data to practice 

Dean Goldhammer concluded his presentation with a challenge for the 
participants to exercise leadership in needed changes in the bases of 
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decision making. He conjectured that Planning, Programming, Budgeting 
Systems could accomplish the foregoing. 

Mr. Cecil Stanley, Director, Division of Vocational-Technical 
Education, Eebraska, gave the next two presentations. The first 
focussed on state level administrative problems in vocational- 
technical education. The second focussed on financial problems at the 
state-federal level in vocational education. The purpose of these 
topics was to help set the need for application of PPBS. Mr. Stanley s 
presentations were from the view of a state director. The speaker 
elucidated on most of the problems stated, but they will be listed 
simply as follows: 

1) How to develop a state master plan with all the planning 
implications ? 

2) Relationship between philosophy, goals, and objectives and how 
they can be validly established. 

3) What should be the organizational base for post-high school 
vocational and technical education? 

4) What should we do about vocational education at the secondary 
school level? How can we resolve this issue? 

5) How can miltilevel programs be efficiently organized and 
articulated? Is there a method to determine how to solve 
this problem most economically? 

6) What is cooperative education and how can it best be organized? 

7) What real effect does federal legislation and red tape have on 
state programs? Can these be minimized? 

8) What is an effective mix of interdisciplinary training in 
vocational education? 

9) How can we effectively broaden the target population base? 

10) What part should the private sector and organized labor play 
in the whole scheme of vocational education? 

11) And what of the ancillary youth organizations such as the 
Future Farmers of America and others? 

12) The entire panorama of evaluation. 

13) Relation of vocational education to general education, community 
colleges, and four-year Colleges and universities. 
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14) Relationships with advisory groups and governing boards. 

15) Coordination with teacher education programs. 

In closing, Mr. Stanley suggested that FPBS should definitely 
assist in solving many the problems discussed and hoped that it 
would solve all of them. 

Mr. Stanley’s second presentation was "State and Federal Finance 
Problems in Vocational-Technical Education." He addressed the problem 
of establishing priorities of expenditure with the limited funds usually 
available. This emphasis focused sharply toward the entire Institute 
program as a major purpose of PPBS is the effective allocation of 
scarce resources. He reased the following questions with respect to 
the problem: 

1) Can PPBS, or any other system, help set priorities? 

2) On* what defensible bases? 

3) Can the system be applied to the oldtv.*, traditional programs? 

4) Should an organization begin PPBS with new programs only? 

5) Can PPBS establish the cutoff place for pumping money into 
any particular program? 

6) Few can needs be validated with hard data? 

7) What kind of research and development effort is needed to apply 
PPB systems? 

8) Can simpler systems of reimbursement (financial aid distribution 
formulae) be developed out of the system? 

9) Can workload units be validly defined out of the collectable 
data utilized in applying PPBS? 

A Panel discussion concluded the program day. See the program in 
Appendix A for panel participants. The Panel’s assignment was to discuss 
any and/or all of the major quest ions /problems raised thus far by the 
two speakers. Major issues discussed included: 

1) Research data, such as task analysis, manpower needs, which 
could help make vocational education relevant, with specific 
reference to Dean Goldhammer’s opening address. 

2) Dr. Worthington, Assistant Commissioner for Vocational Education 
in New Jersey, outlined how he hoped that PPBS could assist in 
making decisions in selecting from ten alternative proposals 
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in New Jersey. 



3) Multiple budgets and the addition of a PPBS budget was raised 
as an issue. Is there a way out of the multiple budget 
dilemma? 

4) What should be included for reimbursement in vocational 
education? The need to define the cost factors was emphasized. 

5) Dr. Otto Legg, USOE, stated that he thought that analysis 
systems was the problem and asked the question, "Can a segment 
of vocational education be analyzed for baseline data?" 

6) Several studies defining manpower requirement needs were cited 
as necessary data for valid program decisions. 

7) The issue of new systems for reimbursement was discussed at 
some length. Dr. Worthington suggested program reimbursement 
and a subsequent delegation of authority rather than a 
concentration of decision making at higher levels. 

The second day began with a presentation, "Need for PPBS at - che 
State Level," by Mr. Arnold Cogan, State Planning Coordinator, Governor* s 
Office, State of Oregon. Mr. Cogan quite naturally emphasized the 
Planning potential and strength of PPB. He began by defining the 
purpose of planning as assisting in decision making. Oregon* s major 
planning activities in government are done as a part of the Governor *s 
office and is in keeping with the trend in state government and 
management. The rationale given was - that the resulting decision making 
at this level "has the muscle" to accomplish the planning task and 
implement the results. Both long and short range planning should be 
done on hard data and accurate interpretation. A system of allocating 
scarce resources must be established as there is never enough money to 
to around. Mr. Cogan emphasized the need for a multi-discipline approach 
in planning. He feels that the specialist tends to become myopic and 
sometimes fanatic. (A fanatic was defined as someone who redoubles his 
efforts when he loses sight of his objectives). 

Mr. Cogan suggested the task force organization in attacking 
planning problems. He emphasized the importance of defining objectives 
and their function with respect to developing program structure and 
organization. Mr. Cogan closed with a brief resume of Oregon *s state 
government experience, with implementing PPBS to date. He suggested 
establishing demographic regions in a state as a basis for planning and 
re— emphasized that the federal government could not establish the planning 
effort necessary to solve the states* problems. 

The next major program input was a double-session presentation of 
an overview of Planning, Programming, & Budgeting Systems. Presentor 
was Mr. Lavor Neuenswander, Director, Financial Management and PPBS 



Training Institute, Bureau of Training, U. S. Civil Service Commission. 
His presentation is outlined as follows: 

1) Identification of long range goals and short term objectives 
with the philosophy and purposes of the country, state, and local unit 
of society (government). Explication of program objectives in terms of 
quantifiable ouptputs is necessary for evaluation and analysis. 

2) Definition of the program structure with respect to the goals 
and objectives. Preparation of the program memoranda based on the 
defined program structure. 

3) Preparation of the program financial plan on both the financial 
and real resource cost bases. 

4) Projection of costs for five years. 

5) Application of various ievels of analyses including cost 
benefit, cost effectiveness, and others. 

6) Planning evaluation system and corrective feedback system. 

Mr. Neuenswander pointed out that the PPB system costed out programs 
designed to achieve national (or other) goals. Implications of PPBS 
for administration includes possibly reorganizing to fit the program 
structure, keener competition for available funds, and realignment of 
accounting systems to fit the program structure. (This would be a GAO 
auditors utopia). He also opined that making appropriations on a 
purely, political basis became more difficult with the PPB system. PPS 
does not make decisions for the decision-maker, but orders the data so 
that more effective allocation of scarce resources (decisions) can be 
made. He stressed the desperate need for developing an adequate data 
information system and to train competent analysts if one is to 
implement the PPB system. Mr. Neuenswander indicated that, in his 
opinion, the PPBS organization should serve as the right hand to the 
decision maker. He closed with remarks concerning operation of PPBS 
at various levels of an organization and that much adaptation and 
development was needed and easily possible. 

An additional session was spent in a question and answer session on 
the various technical aspects of PPBS and the variety of problems 
arising in the implementation stages. Availability of training materials 
and opportunities was also discussed. 

The Institute Program now turned its attention to detailed 
instruction on the various aspects of PPBS which were needed if partici- 
pants were to understand and be able to implement the system. 

The next presentation was on "Clarifying Philosophy, Goals, and 
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Objectives,” presented by Dr. Gerald Gage, Research Professor, Teaching 
Research Division, Oregon State System of Higher Education. The following 

is a direct quotation: 

"Objectives are statements which should follow the recognition of 
needs or the establishing of some priority of desires. If something 
can be classified as a need, the argument as to whether or not it should 
be met has largely been answered. To the extent any person or society 
does not have its needs met, it is lesser person or society than it 

could be. 

There are times when it would be desirable to have something 
even though it isn’t necessary to have it. We do appreciate air 
conditioning, wrist watches, and programs in vocational education. 
Occasionally, these come to represent conventional necessities which o 
improve human society. Individuals and society would be lesser without 
them. These should not be confused with whims. Whims can provide us 
with pleasure, but these objects are usually transient and in their 
absence we get along without them without particular discomfort. In 
spite of the inference in the television commercial, the loss of BAN 
deodorant would have little effect upon society. 

I’m not going to do much to develop the concept of sources of 
objectives. One thing should be recognized; even needs are not always 
easily identified. For example, the student perceived as not college 
bound" has the same need for successful learning experiences as the one 
who is perceived as "college bound," yet this need has gone largely 
unheeded. Yet, once we agree on a need, society generally attempts 
to meet it. A society which fails to meet the needs of its members is 
usually replaced by one which does, or at least promises it. 

The way we respond to needs, desires, and whims depends primarily 
on our view of man. This gets into philosophy which is another topic 
albeit related. The point is that if you are going to develop objectives 
you should find out what exists and is needed. Yet many educators 
generate programs with no concept of whether they (the programs) are 
concerned with needs, desires, or whims. I suspect that the need 
being met is the one they (the educators) have for a job so a 
is established which does little except provide those m it something t 
do. Certainly State Departments of Vocational Education are not different 
than other bureaucracies. Many of the staff are there to car ^°^ t 
functions which never leave the office nor do they change anything within. 
Of course none of you engage in such activities. We at Teaching Research 

do. 

At any rate, don't try to state objectives until you have identified 
, needs and/or provided some evidences of the consequences of meeting a 

desire. 
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There is something about the words ’educational objective 1 which 
appeals to the intellect and emotions of professional educators* The 
appeal isn’t quite sufficient that most of them spend much time developing 
statements of objectives, but, at least as a dogma, the concept of 
objectives has become sainred as Mager points out in his parable of the 
seahorse who swam into the shark’s mouth in search for fame and forture, 
’...if you’re not sure where you’re going, you’re liable to end up some- 
place else.* No educator wants to be accused of not knowing where he is 
going or of imitating Columbus, that is, not knowing where he was when 
he got there. 

Just as the educator doesn’t appreciate the accusation that he 
doesn’t know where he is going, he doesn’t want it said he can’t 
communicate to others where he is going. So the challenge in construct- 
ing objectives is to make them clear. If they are stated clearly 1) the 
planner can state exactly what he expects, 2) the operator knows exactly 
what he is to do, 3) the budget officer can state exactly what it costs, 
and 4) the evaluator can tell whether a particular method is having the 
desired ef fecti. , The only objectives which accomplish these things 
are those which are stated behaviorally. The concept of behavioral 
objectives is not a new one to you but I suspect we need to have con- 
siderable information about them before we can use that concept. 

In order to implement the concept of behavioral objectives we need 
to think about the criteria which distinguish a behavioral or performance 
objective from one not stated in behavioral or performance terms. I’ll 
discuss this with you Tuesday afternoon." 

After the foregoing introduction. Dr. Gage dwelt on the following. 

He stated that people’s intentions (to accomplish something) imply some 
outcomes (outputs/objectives) and strategies of achievement. The 
sources of our intentions are in law, needs, desires, and whim. A 
criterion for an objective rests on its worthiness and clarity. Worthi- 
ness probably must be decided on a value system; clarity can be developed 
in an objective on a specified basis. A paraphrasing of Dr. Gage’s 
presentation results in the four major characteristics of a properly 
specified objective. These may be termed the ABCD’s of Objectives. 

1) Audience or target population. 

2) Behavior or desired, skills, knowledge, attitudes. 

3) Conditions of learning such as the setting, context conditions. 

4) Degree of learning beginning with minimum level acceptable 
and reasonable and higher levels possible in clearly defined 
steps. 

Dr. Gage emphasized that the foregoing outline resulted in a 
measureable criterion which was necessary if proper evaluation was to 
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evolve. The "direct" and "indirect" nature of objectives was discussed 
with particular reference to difficulty, if not impossibility, of 
evaluation efforts if objectives were "second and thirdhand" removed 
from the program activity. 



Buzz groups were organized and given the following assignment for 
Wednesday, August 21st. 



Instructions to Group Sessions on 
Quantifying Objectives ' 



1) What are the sources for your objectives? 

2) What are the criteria for your objectives? 

3) Define the differences between philosophy, goals, and objectives. 

4) What are some valid objectives of your state level agency in 
quantifiable terms? 

5) What difficulties did you have in answering the foregoing? 

6) Please have one member of the group prepare a brief oral report 
and hand in a written report. 



Each buzz group selected a leader and reported to the larger 
group. The reports followed the outline of the assignments and led to 
much discussion and opportunity for further clarification by Dr. Gage. 

He then, utilized examples of specifying objectives for the instructional 
sessions on objectives. One is stated as follows: 

OBJECTIVES 

On a cest, participants in the PPBS Institute can: 



1) List two sources for objectives. 

2) List four characteristics which determine whether or not an 
objective is clearly stated. 

3) Identify the characteristic (s) missing in an incompletely 
stated objective. 

4) Identify an objective which contains all the suggested 
characteristics . 
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5) List the two main criteria for evaluating an objective* 



The sessions on Objectives were concluded as Dr* Cage administered 
the following test: 

TEST ON OBJECTIVES 

1) List the four characteristics which are part of a clearly 
stated educational objectives. 

Here are four objectives written for students enrolled in Social 
Studies, grade 5. The students should: 

a) Describe the way horses were used by pioneers in the northwest 
United States before 1900. 

b) List five products. 

c) Label a minimum of thirty-five of forty-eight mainland states 
on an outline map of the United States. 

d) Appreciate the importance of history in understanding the 
Pacific Coast Indians. 

2) Which characteristics are not present in the first objective? 

3) Which characteristics are not present in the second objective? 

A) Which characteristics are not present in the third objective? 

5) Which characteristics are not present in objective number four? 

6) List two sources of objectives. 

7) List the two main criteria for evaluating an objective. 

8) Write one objective for your Department of Vocational Education 
which meets the criteria of a behavioral objective. 



The next two sessions were spent on the topics of ''Developing the 
Program Memoranda, Terms and Definitions." The Institute Director 
served as instructor for these sessions. His presentation is paraphrased. 

The speaker introduced his topic by stressing the need for 
definitive terminology. No apologies should be made for particularized 



nomenclature which is necessary for clarity of communication in any 
field. Much clarification of terminology is needed in any developing 
field and PPB is no exception. 

A "vehicle is needed" in which to transport the PPBS message. This 
vehicle is the program memoranda (PM) . What is a PM? The program 
memorandum is the document for an institution or agency, which includes 
the statement of purposes, ana objectives, description of programs, 
budgetary and other analytical data, and the recommendations. The 
speaker conceives of the PM as a broadly based, inclusive type of 
document, rather than a simple program description. It is a vehicle for 
ordering information to assist the decision maker in arriving at 
rational decisions. In jargon closest to current usage, it is the 
budget document. The inclusive, broad aspect of the foregoing definition 
is valuable in focusing on a primary goal of PPB systems — decision 
making based on program. 

Dr. McAbee next outlined the elements of the program memoranda . 

1) General statement of purposes and long range goals of the 
agency beginning with needs. 

2) Sources of tasks (programs) assigned, assumed, or legally 
mandated. 

3) Reasons for the general and specified needs. 

4) Particular issues involved. 

5) Assumptions on which the PM rationale is based. 

6) Specific objectives for the various program categories (output). 

7) Program categories, sub-categories, and elements defined. 

8) Alternative programs. 

9) Analyses including the cost effectiveness and cost benefit 
levels . 

10) The program financial plan (PFP) for the recommended programs. 

11) Projection of future costs. 

12) Strategy underlying the recommendations. 

13) Estimation of available resources. 

14) Assessment of the political factors involved. 



A number of definitions were utilized during the presentation. 

These included: 

Alternative programs are programs designed to attain the same 
objectives as the origianl program. 

Goals emanate from statements of belief and philosophy. They 

constitute longer range attainment in contrast to objectives. 
They are also more general in nature. 

Obj ectives consist of statements which can be utilized as measurable 
criterion. A properly specified objective includes the target 
population, behaviors desired, content, context, or setting 
for the program, and degree of attainment. 

Cost effectiveness is the cost per program unit produced. 

Cost benefit is the relationship between the cost of the product 
and value of the benefit of producing it. It may be expressed 
as benefit-cost . 

Output is a unit of product or service of the program activity. 

Technically, it is what is produced — the means of achieving the 
objective. 

Needs are the goods and services required for the well-being or 
•improvement of the citizenry in society. 

PPBS is a systems approach to defining, collecting, and analyzing 
data, which can be logically brought to focus on the problem 
of allocation of resources for the purpose of improving the 
effectiveness of the decision maker in maximizing the benefits 
for the money expended. It involves the definition of needs, 
goals and objectives. Definition of program structure, cate- 
gories, and elements follow. Alternative strategies are de- 
veloped to meet the objectives and an analysis of alternatives 
presented. Analysis in PPB varies from cost-benefit, considered 
by most to be of the highest sophistication, to mere unsupported 
opinion or prejudice of the decision maker. 

A variety of additional sources of definitions are available. 
Probably the most complete in "Planning, Programming, Budgeting Systems 
Analysis GLOSSARY," published in January, 1968, by the U. S. Accounting 
Office. 




Thursday, August 24. 

Dr. Otto Legg, Assistant Director, Program Planning Division, 

Bureau of Vocational-Technical tducation, U. S. Office of education, 
gave s presentation on the status of PPBS with respect to USOE view- 
point, "Planning and System Analysis, (PPBS) Development in Vocational- 
Technical Education." Dr. Legg f s remarks focused on planning and 
systems analysis, evaluation and presenting evidence of program worth 
in an understandable fashion. Systematic planning is demanded to 
obtain a better balance of program with due consideration to the 
needs of the people and resources available. He stressed the scope of 
the task in vocational education to be threefold; initial training, 
retraining, and continued upgrading. Comprehensive planning, including 
alternative programs, is necessary to fulfill the needs. Broad planning 
should also include non-education agencies and resources. Dr. Legg 
next defined the sources and nature of agency mission, long range 
goals, and short term, measurable objectives and the necessity for these 
to be consistent (reliable). The objectives must be in a time frame 
and otherwise quantified in order that degree of attainment is 
measurable. These, then, serve as a blueprint for action and define 
the programs. 

The speaker next emphasized that analysis provides the information 
but not the decision. Analysis must begin with the problem definition. 
The necessity and scope of data requi.red for analyses were discussed. 

The need for new types of data in order to accurately forecast needs, 
programs, and costs was shown to be an expansion of traditional planning 
patterns. 

The dangers of confusion in defining problems with new dimensions 
was stated. Changing value systems (evaluation bases) may further 
confuse the definition and palnning effort. A combination of accepted 
values and facts are usually necessary for acceptability to the decision 
makers . 

Planning must also include valuation if it is to be complete. Dr. 
Legg defined program evaluation as follows; "Program evaluation is a 
process of determining the extent to which predetermined objectives and 
predetermined levels of operation are obtained. ” Evaluation must provide 
the corrective feedback to program as well as furnishing the measure of 
success or failure. As long as change takes place, educational leaders 
must increase the attention to a systematic process of planning that 
change. 

Dr. Legg pointed out that, in the increasing competition for funds, 
more sophisticated means of analysis (cost benefit and others) must be 
developed if we are to convince the decision makers that educational 
requests are valid. Research efforts and program operation must proceed 
together if decision making is to become more effective in helping the 
field of education meet the needs and achieve its objectives. 
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The various buzz groups next reported on the previous assignment, 
"Defining Data Needs in PPBS for the Program Memoranda." These reports 
resulted from the following assignment: 

1) Develop a PM outline for a selected real or hypothetical 
educational institute. 

2) Fill in the outline, as far as time permits, with a list 
of data which is needed to complete the PM. 

3) Compare the data needs with what you know to be usually 
available. List those items about which you would need 
additional details. 

4) List any technical vocabulary which needs clarification. 

5) Have one member of the group prepare a brief oral and 
written report for Thursday 1 s session. 

This session was directed toward assisting the participants to 
face the questions which must be answered in each situation as a step 
in implementing PPBS. After the buzz group reports, an individual 
"s^tudy and think" session was scheduled for focusing on the partic- 
ularized situations represented by each participant. This was a diffi- 
cult assignment at this stage of the Institute and it was planned to 
be. Questions and problems raised did much to develop readiness for 
subsequent Institute topics, presentation and study. 

Dr. William Frei thaler. Department of Economics, University of 
Virginia, and frequent instructor on PPBS topics for the Bureau of 
Training, Federal Civil Service Commission, was the next major speaker. 
Dr. Freithaler*s assignment was to cover the basic economic and 
statistical principles in PPBS, followed by presentations of cort 
effectiveness and cost benefit analysis. 

He explained briefly, with numberous examples and statistical 
illustration, the following economic concepts. He illustrated their 
application to vocational education. 

1) The Phillips Curve Concept 

2) Law of supply and demand 

3) Law of diminishing returns 

4) Marginal analysis 

5) Macro vs. micro economic theory 
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6) Indifference curves 

7) Regression analysis 

8) Theory of Discounting 

9) Dealing with Uncertainties 

The major point was that PPBS emanates from classical, economic 
principles of the capitalistic society and thereby furnish a valid 
base to the economics of education and the various analyses utilized 
in the PPB system. 



Friday, August 23. 

"Introduction to Cost Effectiveness Analysis" 
presented by Dr. Freithaler 

The first task in analysis is to define the variables to be costed. 
These include both independent and dependent variables. A dependent 
variable would be wages, which vary according to sex, age, education, 
and location of the wage earner. An independent variable would be the 
wage as reflected in the choice of job level of which the worker is 
capable . 

As early decision in analysis must be made as one of two basic 
approaches : (1) the direct financial or out —of —pocket costs of (2) 

the real resource or total economic costs. Dr. Freithaler pointed out 
that the first would result in the amounts to be actually budgeted and 
spent and the second approach would be the toatl, direct and indirect, 
costs to the institution, the individuals in the program, and society in 
general. The real resource cost bases should be utitlized primarily in 
cost benefit analysis and the financial approach to cost effectiveness 
analysis • Cost benefit was defined as the ratio derived by dividing the 
value of the benefits by the costs. He briefly outlined the need for 
adequate data streams for various types of analysis • The implication 
for cost accounting systems, developing analysis for alternative programs, 
and necessity for evaluation systems in order to measure success /failure 
and thereby to really ascertain the effectiveness in dollar terms were 
also developed. He suggested a simple amortization plan for assigning 
facility costs to output units. Support service costs can be handled 
in the same manner — dividing total support service costs by the FTE 
(full time equivalent) in educational situations. 

With the foregoing overview. Dr. Freithaler next turned to a case 
study application in education of the principles • This case was taken 
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from “Cost Analysis for Planning-Programming-Budgeting Cost-Benefit 
Studies," by J. D. McCullough and published by the Rand Corporation. 

The instructor led the participants through this case step-by-step 
with much question and answer discussion. Unfortunately, the McCullough 
Case is too lengthy to insert in this report; however, it is recom- 
mended as essential training materials and can be easily secured (see 
Bibliography in Appendix C). 



A buzz group was next on the schedule. The topic was "Potentials 
for Research and Development in PPBS." The following assignment was 
given: 

1) Please address yourself particularly to research potential 
in cost-benefit and cost effectiveness analyses which would 
furnish baseline data which might have a general application. 

2) List the foregoing and submit a brief oral and written report. 

This exercise produced a number of suggestions for research and 
development needed in vocational education for the PPBS. These included 
development of PPBS applications for various levels of application, i.e., 
state divisions, federal office, schools, departments, single program 
elements. Baseline data are needed in manpower requirements, manpower 
characteristics and availability, and program costs based on comparable 
models. Data leading to a criteria for cost effectiveness and cost bene- 
fit ratios are desparately needed. Models for assessing the foregoing 
factors, and especially the value of benefits, are needed. As to pricing 
benefits, increased income and pricing spillover benefits are essential 
data needs and represent the greatest vacuum in PPBS data requirements. 
Evaluation models were designated as a “must" need if application of 
PPBS was to fulfill its potential. 



The first week's program concluded with a "Review of the PPBS 
Structure" by the Institute Director. This review included the fol- 
lowing items: 

1) Overall review of PPBS. 

2) PPBS is a system for planning, management, and control. 

3) It is based on sound economic principles. 

4) Furnishes the basis for a goal contered operation. 

5) It is adaptable to several levels of operation. 

6) PPBS is a vehicle for organizing data to improve decision 
making. 
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7) It maximizes the results for the dollar expended. 

8) PPBS can be used with a variety of analytic techniques. 

9) Provides a basis for generating and comparing alternative 
programs . 

10) PPBS provides for evaluation and corrective feedback. 

11) Projects expenditures over time. 

12) Provides for stipulation of issues and priorities in program. 



Monday, August 26. 

"Planning Cost Analysis of a Training Program," 
presented by the Institute Director, Dr. McAbee 



In my opinion, the area of cost analysis is the area in which much 
research and development work needs to be done if PPBS is to fulfill its 
potential. This is, maximizing returns on the investment in consonance 
with need priority. The need priority has a vital role in consideration 
of alternative programs as the most economical (cheapest per unit) may 
not meet the greatest need. There are two other essential, primary con- 
siderations: (1) should a real resource (total expenditure) or financial 

approach be utilized in analysis of costs and benefits? ; (2) how are the 
terms defined? There is much obfuscation in PPB terminology. One per- 
uses the literature and might easily assume that cost utility, cost 
effectiveness, and cost benefit studies are synonomous terms. They are 
not! (For current purposes, cost benefit and cost effectiveness defin- 
itions may be found in an earlier presentation on Program Memoranda and 
Definition of terms) . 

Several issues present themselves in computing cost benefit ratios 
for various programs. The first, what costs should be considered? The 
primary issue here is the financial (direct) vs. real resource base. 

The real resource approach should be used in cost benefit analysis in 
order to compute price of values or benefits. These values are usually 
not recoverable or accrue to the institution or agency and therefore, 
cannot be utilized in the operational budget. What benefits (values) 
should be included? It is difficult at best to assign dollar values to 
some benefits. Some benefits, such as increased income stream are rel- 
atively easy to compute. Others, such as the wide variety of so-called 
"spillover" values, are more difficult and frequently, impossible. The 
analyst cannot accurately foresee everything in the future and place a 
price tag on it. There are three or four kinds of values which help 
in getting a start. These include: 
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1) Value of increased income. 

2) Value of increased productivity. 

3) Spillover benefits such as decreased welfare and crime rates, 
and increased health status. 

4) Value of residual educability. (Residual educability is the 
added education potential accruing to an individual which is 
attributable to completion of a program). 

As you will note there are not readily available data on many of these 
items. A massive research effort is needed. However, a persevering 
analyst will get at some of these, especially the increased income 
concept. 



In cost effectiveness studies, we are trying to get at the unit 
cost, cost per output. First, we again must determine the basic 
approach, real resource or financial cost basis. Secondly, we must 
list the costs to be assessed, make certain they are computed on a 
reasonable basis, and use the same model or system in computing different 
program costs for comparison purposes. 

Direct financial costs may be listed simply as: personnel, supplies 

and materials, facilities and equipment, supporting services such as ad- 
ministration, counseling and maintenance. The student may also have 
some direct costs, which might not be in an agency budget, such as lunches, 
transportation, clothing, etc. 

In addition to these, a real resource approach calls for the ad- 
dition of opportunity costs, transfer costs, foregone, and implicit 
costs. The values of the benefits must also reflect these items. 

A criteria for deciding on what costs to include and how to com- 
pute them will depend on the following factors: 

1) The level of operation, i.e., state agency, school or college, 
department. 

2) It is acceptable or credible? 

3) Can anyone in the shop do the computation? 

Host agencies already have clarified systems for determining direct, 
financial costs. Indirect and real resource costs will usually have to 
be added to these in cost benefit analysis. 

The principle of discounting rears its ugly head at this point. 

You will recognize that the dollar value of some of the costs and most 
of the benefits in cost benefit analysis is at some point in future time. 
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In other words, when we ascribe the dollar value/benefit of a given 
program, we are talking about a value which will probably accrue or 
exist in the future. Now, the current dollar does not have the same 
value as a future dollar— it is much less depending on the time span 
and current price of money. The current dollar is not as valuable 
as a future dollar because people won’t pay as much for it* Go down 
to the bank and ask them! Ask them the price of a $1,000 with deliv- 
ery specified in 1975. It won't be $1,000 current, it will be *- ss * 

And the reverse operates also, if you buy a $1,000 now, it will e 
worth more at some future date. 

Now, in order to present an accurate computation with respect to 
a cost benefit or other analyses, we must correct current values of 
dollars, etc., by discounting and annutizing so that future values are 
reflected into today’s terms. These corrections should be applied to 
both future costs and benefits in order to get the future dollar values 

in today’s terms. 

Discussion followed the presentation and the following assignment 
was given for the succeeding buzz group session. 



Planning An Analysis for 
Vocational Training Programs 



1) Select an existing program for analysis. 

2) Decide whether to base your analysis on real resource or 
strictly financial costs. 

3) List the objectives of the program. 

A) List the individual items of expense you would include in the 
analysis . 

5) List three alternative programs which could conceivably meet 
the objectives. 

6) Are there data available in your shop needed for the analysis? 
If the answer is "no" please list what is needed. 



The various buzz groups reported at a general session. There were 
major problems on prorating facility, equipment, and support service 
costs in training programs. It was suggested that these be simply 
prorated on an PTE (full time equivalent student or teacher) basis. The 
need for an adequate data stream was emphasized as most reporting groups 
had long lists of data not currently available, which would be nee e 
in these computations. The factors of real resource and financi al costs 
were discussed also. The principal clarification needed was the -yP e 
(direct, financial) which makes up the operational budget and the cost g 
approach which arrives at a cost benefit ratio for a program rom an 
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examination of all the costs and all the values to and in society which 
are attributable to the program. 



’’Budget Cycles, Formats, and Crosswalks 
by Harold V. McAbee 

These three related topics are technical in nature, but necessary 
in any type of budget presentation. Application of the PPB system 
lends a new dimension to budget format and leads to the necessity for 
crosswalks . 

Budget Cy cles . Most administrators recognize that they are working 
with about three budgets at any one time. One will be the budget in 
preparation (Planning Stage) . The second will be the current operating 
budget and the third will be the "cash flow" budget. All of these will 
be operating on a different time schedule and seldom will cycles within 
these three budgets become congruent. 

Now, what is a budget cycle— PPB type? An iterative system serves 
to illustrate. An excellent illustration may be seen in "Systems 
Analysis Techniques for Planning-Programming-Budgeting," published in 
1966 by the Rand Corporation. The author is E. S. Quade. 

A simple systems outline also furnishes an excellent illustration 
of a PPBS budget cycle. It follows the following steps: Identify 

problems, analyze problem and set goals, determine solution (program) 
strategy, implement, evaluate, and introduce corrective measures. 

The foregoing may serve as the basis for establishing a calendar 
of dates and deadlines so that the work for the entire budget cycle may 
be planned and timed. Begin at the future date represented by submission 
deadlines and use backtiming throughout to establish a calendar through 
the entire clcle. This is especially important in a PPBS budget cycle 
because it will probably involve a long time span than represented in a 
legally designated, fiscal period budgets. An example of a budget 
calendar follows. All well managed shops should develop one. 



CALENDAR 

BUDGET 1968-69 



1967 


December 


12 


Forms issued to heads of departments 


1968 


February 


12 


Budget estimates due in Business Office 




19 


Budget conferences with heads of departments and 
Superintendent 
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March 


19 


Preliminary budget estimates to Board 


April 


n 


Preliminary budget explanation of increases and 
decreases by items to the Board 




16 


Revision of budget after all salaries fixed 


May 


7 


Preliminary budget explanation showing estimated 
levy 




7 


Board discussion of budget 




21 


Board discussion of budget 




21 


Preliminary sets of budget to the Board 


June 


4 


Completed budget book to Board 




18 


Budget and site levy adopted by Board 




20 


Budget publication 


July 


8 


Public hearing on budget; application to increase 
levy adopted by Board 




10 


Budget copies distributed to staff and other employees 


Augus t 


9 


Budget certified to auditors in Poik and Warren 
Counties 




10 


Budget bound (file copies) 



Budget Format presents another primary problem in a PPBS system. 
Most traditional type budgets list the inputs as budget outputs, i^e., 
teachers appear as objects (outputs) rather than inputs. This differ 
entiation must be made in a PP budget because of the specification of 
objectives in terms of outputs, which are the measurable criterion and 
quantitative in nature. But these represent end results to be reached 
as contrasted with metnods-means (input) items. 

A PPBS budget format (Called a PFP— program financial plan) will 
be in terms of input and output items arranged according to the program 
structure. This format will almost always be different than currently 
budget forms. Expenditures will also be projected as will data on needs 
and inputs. A suggested projection is three-five years. In the real 
world it will probably be only for the fiscal period immediately ahead. 
The State of Wisconsin has had several years experience with the format 
problem and have developed format for an excellent budget document. 

Write to Mr. Clarence Greider, Director, Vocational, Technical, and 

- 29 - 



rmp. 



Adult Education, Madison, Wisconsin, for a copy. Because of variance 
in state and local organisations and degrees of extertise, most agencies 
will probably develop its own format. 

Crosswalks . One of the most perplexing problems encountered by 
PPBS implementers is the problem of interpreting the newer, PPBS 
formats to accountants, fiscal agencies, and legislators. When budget 
inputs and program objectives are properly specified and arranged, the 
resulting figures are not comparable to former authorization or 
accounting formats. It is likely that legislators and accountants 
will not like the new approach. At this time when educators are 
requesting ever increasing sums of money, the new format may well be 
interpreted as an attempt to confuse by some legerdermain and thus 
obtain appropriations not otherwise forthcoming. The accountants get 
upset because they have to change their account forms, which in my 
experience, accountants are usually loath to do. Given the usual power 
which financial types often wield in an organization, it behooves the 
PPBS implementer to sidestep this obstacle. It can be done very simply. 
Translate the PPBS budget into authorization format or the accountant’s 
terras once the original analysis has been completed. If the agency has 
an EDP bookkeeping system, an infinite number of budget formats can 
be plugged in and retreived. If the accounting is done manually, it 
is suggested that the crosswalks (translations) from the PPBS format to 
the accountants format be performed during the development of the PFP. 

There is considerable work under way toward solutions to these 
problems. Much more needs to be done. Watch your professional and 
USOE publications for developments. One last admonition - a PFP must 
be developed for various levels of operation. A federal format will 
not likely fit state or local needs and vice versa. A state agency 
format will not usually furnish a total solution for the local district. 



Tuesday, August 27. 

"Discussion of Case Study: Benefit-Cost 

Estimates for Job Corps and Implications 
for State Level Vocational Education" 

This case study was prepared for the Office of Economic Opportunity 
by Glen G. Cain, Department of Economics and Institute for Research on 
Poverty, University of Wisconsin. This was an actual case, originally 
prepared for 0E0 purposes, and has been utilized as a training aid for 
the Bureau of Training, Federal Civil Service Commission and others. 

The case served to illustrate a number of difficulties encountered 
in initial attempts in applying cost benefit analysis. Some of these are 
scarcity of valid data, lack of precedent in utilizing this approach. 
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problems in the real resource approach, credibility and acceptance. The 
paper is a pioneering effort in applying cost benefit analysis to 
educational programs. 

Participants had read the case and the Institute Director served 
as discussion leader in the discussion. The following points were 
highlighted which have direct and indirect implication for applying 
cost benefit studies to vocational-technical education: 

1) Some intangibles present difficulties for measurement. What 
does one do about these? Secure added data, forget them, handle 
with statistical methods dealing with uncertainties. 

2) A variety of characteristics of the target population, including 
place of residence, will affect the accuracy of some of the data 
and baseline measurement. 

3) Cost data must frequently be brought up to date and the 
principle of discounting reapplied. 

4) What discount rate needs to be applied? My answer was to use 
rate currently needed to finance projects for the institution 
for the program period involved. If it is for one year, the 
banks can easily tell you their bid price to finance a short 
term program. If the analyst Is projecting costs asid benefits 
fifteen years in the future, the price of government bonds 
maturing at that time, might be a proper rate. 

5) When the total costs are compiled, one usually learns that 
training is a good deal more expensive than first examination 
reveals . 

6) Costing analysis must reflect the initial “ startup” or research 
and development costs separately . If these are charged to 
operating, unit. costs become distorted upward. In new programs, 
this factor will be significant. 

7) Capital outlay costs must be prorated and amortized. 

8) A way must also be found to prorate support service costs. 

This is a major problem for state agencies which specify 
student programs as objectives. The stage agency costs must 
then be distributed or prorated to the districts operating the 
programs, or not charged at all, which distorts the real and 
actual costs downward. For agencies operating programs, a fair 
and simple way is to divide the support costs by the FTE (full 
time equivalent of students, teachers, or perhaps the number 

of contracts or districts). 

9) So-called transfer costs must be carefully handled. It is easy 
to impute equal values to two entities which are not really 
comparable. 
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What kind of cost-benefit ratio constitutes a "good" 
(acceptable) one? Few data are available, but one does know 
that if the bene --- t - » less than 1., the program won’t pay off. 
Much resear ch C needs to be done in this field. It is likely 
that the standard for an acceptable cost benefit ratio would 
vary in different agencies, institutions, locations, types of 
programs, and quality of programs. 



"Generating Alternative Programs" constituted the next buzz group 
assignment. Although the generation of alternative programs to meet 
the same objectives is an essential part of PPBS, few agencies, 
especially in education, have made consideration of alternatives 
operational. 

There are a number of standard techniques which can be utilized 
in the alternative program effort. One is to vary the richness of the 
program by changing the time span involved-, the number of target 
population, teachers, materials, and appropriate but added applied work 
experience. Such changes can be costed and the resultant change in 
cost benefit ratios computed. Evaluation of results can determine 
whether any of these variations actually resulted in an improved 
effectiveness and cost benefit ratio. A second major method of altern- 
atives is to change the target population and probably, but not neces- 
sarily, the program. 

Another major means of consideration of alternatives is to trans- 
fer the activity from the public to the private sector . This may be 
bold in conception and grate harshly on: public-type mentalities, but 
the fact remains that the private sector does a tremendous amount of 
successful training and convinces people to pay for it and they don’t 
stay in business if they don’t make a profit. 

The essential in valid analysis and choice between alternatives 
rests with properly specified objectives and congruency with goals. 



The buzz group assignment follows: 

Instructions to Group Sessions 
on Generating Alternative Programs 

1) Identify three types of training programs for which your 
agency has some responsibility. 

2) Define the quantifiable objectives for these three programs . 

3) List at least three alternative methods of reaching the 
program objectives. 
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the comparative costs of each 



4) Indicate the group concensus on 
of the three programs. 

5) Indicate the group concensus on the comparative benefits. 

Most of the responses and discussion from the buzz groups were on 
a philosophical level. Mainly, does a state level agency have the 
flexibility to generate alternative groups and why should the private 
sector get into the picture? 



Case Study : ’’Cost Benefit and Cost Effectiveness Analysis - 

Development of HEW Disease Control Case. 

The use of this contrived case was designed to review the principles 
involved in analysis as actually applied. The case is a contrived, 
rather than actual, and has some builtin fallacies, the purpose being to 
have participants attempt to spot these as an analyst. The fallacies 
focused on general credibility with respect to the value of the benefits, 
validity of seme projected data, and quesionable assumptions as to 
outcomes of the program. 

A Benefit-Cost Pr oblem was next assigned. This Case was designed 
by DrV Joseph McGivney and Mr. William Nelson, Ohio State University. 

All participants worked through the problem. A number of suggestions 
were made for modification and forwarded to the authors. The Case may 

be found in Appendix F. 



Wednesday, August 28, 



"Levels and Types of Analyses 
presented by Mr. James V. DeLong 



Mr. James V. DeLong is an Analyst, Program Evaluation Staff, Bureau 
of the Budget, Washington, D. C. 

It is a common fallacy to view PPBS only in the context of cost 
benefit and cost effectiveness analysis and to decide that an agency 
can't get into PPB unless a capability exists to generate these types of 
analvses . Several other levels of analyses exist and should be utxiized 
Selection of the level of analyses to be utilized should be made on the 
basis of the current capability and time constraints. t ® ® f 

and acceptable to begin at lower levels and succeed to higher degrees of 
sophistication as experience and skills increase. 

Mr. DeLong then outlined seven (7) levels or types of analyses. 
These will be given in descending order of sophistication. 



1) Cost benefit analysis. 

2) Cost effectiveness analysis. 

3) Direct cost per unit of output. 

4) Cardinal weighting of advantages and disadvantages of the 
program. 

5) Ordinal weighting or ranking of alternatives. 

6) A simple qualitative rationale for the recommendations. 

7) Simply state why or how the decisions were reached. 

Mr. DeLong*s second presentation was on the "Role of the Budget 
Examiner and Analyzing Needs . M He began by outlining some role gen- 
eralities of the analyst. These included: 

1) Decisions are made on economic terms. 

2) The goal is to maximize value of the dollars spent. 

3) Must differentiate between risks, uncertainties, and assumptions. 

4) Be creative in quantifying the social sciences. 

5) Stimulate development of alternatives. 

6) Improve data collection, development, and interpretation. 

7) Urge development of valid program evaluation. 

8) Insist on accurate use of proper statistical models. 

9) Examine organizations and their management and operational 
programs . 

> 

10) Reduce the requests. 

11) Evaluate legislation. 

12) Question assumptions and objectives. 

13) Cross examine agency representatives. 

14) Play the Devil* s Advocates role. 

15) Assist agencies in their PM development and help them prepare 
for hearings. 

A complete guide, "Program Evaluation Checklist," may be found in 
Appendix G • -34- 



"Problem in Implementation: PPBS in the Washington 

State Division of Vocational Education 11 



Mr. Richard Moe, Assistant Director, was the presentor. He began 
by outlining Washington’s experience with developing a data bank. He 
indicated that a decision was finally made to collect and store all 
data available without too much attention to well defined needs. A 
massage of data raised some startling questions. For example, a close 
scrutiny of the data on training programs in Washington led to a 
re-examination of the objectives cf vocational education. It in- 
dicated a need for administrative reorganization. It enalbed them 
to re-examine uneven distribution of monies by program, population, 
and geography. 

Major problems were encountered in initiating PPBS due to lack 
of direction from the Central Budget Agency. A new budget format had 
to be developed. Adequate data development posed some "hard questions" 
such as, "The State of Washington trains 23,000 secretaries annually 
and has need for 15,000. What do we do with this information?" (Dis- 
cussion among participants revealed that only one or two would be willing 
to reduce the program as a result of these data) . 

A panel discussion concluded the session on implementation prob- 
lems. Several of the participants gave historical accounts of how 
their state came to be interested and involved in PPBS. Two other 
interesting questions arose. One, where should the PPB expert be in 
the administrative organization? Two, does PPBS really move essential 
decision making up or down the administrative hierarchy? The panel 
closed with references to some of the potentials in the PPBS and prob- 
lems of inducing change in state agencies. An ad hoc discussion of 
political factors involved set the stage for the next day’s program. 



Thursday, August 29. Frequent references are found in PPBS literature 
to the political aspects of the entire budgetary process. A similar 
reference will usually arise in discussions concerning the potential of 
PPBS. Reality being somewhat political, it was determined to spend at 
least one day of the Institute on politics .and the budgetary process. 
This c am e to be known as the "political day" at the Institute. A 
variety of speakers were chosen representing several aspects of the 
political arena in education. These included professional educators, 
state legislators, and a state budget director. 

The initial presentation on the political day’s program was made 
by Dr. Angus Rothwell, Executive Director, Educational Coordinating 
Council, State of Wisconsin. Dr. Rothwell* s topic was: "Political 

REalities of the Budget Making Process." His thesis was the "hidden 
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j ii _ ''invisible committee" syndrome operating in current decision 
"SS He detailed thf thesis with a thorough outUne of 

all the aspects of educational decision making in Wisconsin. These 

and ,.,oahl.l O-mr 
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discernible that budgets and politics go hand in hand in the State 
Wisconsin. Dr. Rothwell’s second thesis was that newer ways of 
studies and evolving solutions must be developed. He was quite optimistic 
that PPB systems could help in a substantial manner. 

The next presentation on the "political day" program was, A 
Legislator Looks at Educational Requests," by State Senator Lyna Newb y, 
who is an experienced legislator and has served as Chairman of the Ways 
and Means Subcommittee on Education in the Oregon Legislature. Senat 
Newbryintroduced his topic by briefly reviewing the parameters of 
leeislative authority in considering requests and making app p 
Oregon^s Constitution precludes deficit spending; therefore, every 
legislature must develop a balanced budget with an income constraint and 
unlimited needs, the basic problem for the legislator becomes one of 
priorities. Senator Newbry's posture is that most nee s 
are defensible, but that certain institutions in the ^ucationalarena 
could improve their candor with respect to budgetary details. Senat 
Newbry explained that every legislator, especially if he is °n Way s a 
Means! develops a set of priorities for allocation ofmonies. He was 
willing to share his priorities and they appear as follows. 

1) Wards of the state, i.e., prison inmates, retarded, etc. 

2) Mental health 

3) Welfare 

4) Education 

(There were others but he did not delineate as 
particular need for purposes of this presentation). , . 

?nd£a!ed that the toughest job in setting priorities was within the 

educational category. His appears as follows. 

1) Elementary and secondary education. 

2) Community colleges/vocational education. 

3) Undergraduate higher education. 

4) Graduate and professional higher education. 
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He welcomed the PPB system as having potential for setting priorities 

and opined that it most likely could be applied initially to consider- 
ation of new programs. 

Senator Newbry concluded with some miscellaneous remarks. One, 
existing programs must be examined first when setting priorities. 

Two, educators could help if they would establish some inhouse priorities. 
Three, candidness with respect to budgetary processes somet im es gets 
all mixed up with academic freedom. Higher education ought to do 
better than that. The legislature will probably take increasingly 
closer views at the management of the various state agencies. 

Senator Newbry T s presentation was followed by a panel discussion 
entitled "Improving Communication: A Dialogue Between Professionals, 

Legislators, and the Laity;" Panel members included the two speakers 
of the morning. Dr. Rothwell and Senator Newbry, Dr. Allen Lee of 
Teaching Research, Senator Richard Hoyt, also of Ways and Means, and 
Mr. Cleigh Penwell, Oregon State Budget Director. 

ihe thesis of the panel was how to get effective communication 
between persons of different background on educational matters. There 
was much focus on effective presentations of information using a variety 
of visuals, etc. Senator Hoyt made the major contribution, however. 

His point was that much of the data doesn’t produce much information. 

It is not interpreted meaningfully. Senator Hoyt saw the need in 
communications to be that of ordering data, on a proper conceptual 
base, so that it makes some sense. He thought that PPBS could fill the 
need and "help him do a better job." 

Because of the unique opportunity to consider, in toto, the views 
of a state (Oregon) budget directors toward education and the real 
world, Mr. Penwell s remarks are reproduced exactly as delivered at the 
Oregon PPBS Institute. 

"The Governor T s Office Looks at PPBS 
and Education Requests" 
by Cleigh Penwell, Oregon Budget Director 



f The Education Budget Request. I sort of decided in my own mind I 
didn t want to talk about that very much anyway. I was going to talk 
about something entirely different, but it didn't say anything at all 
about PPB. And so I didn't hear any particular march tc PPB: so I'll 
just go ahead and say what I Intended to and then afterwards when we 
ask questions and have discussions, you can air your questions through 
PPB. Another little problem I had, I made these notes last night, you 
know, so I wheeled the typewriter out there in the family room and the 
Democratic Convention was on. That was quite a show. I was a little 
lste getting finished, and I probably didn't do the job I should have. 
Anyway, with that in mind, we'll go ahead. 
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One thing we’re going to do here in the beginning of this thing 
is sprinkle in a lot of kind of interesting little statistics, mainly 
because I’m fascinated by the statistics when they’re interesting ones 
and I’ve got some here — they really astonish me — and I’ve found that, so 
if I just drop in some type of static that doesn’t seem to have any 
particular relevance to what I’m talking about, you’ll know it’s just 
because it was an interesting one. I’ve got a little book here that I 
stumbled on the other day, and since apparently this is being recorded, 
I’d better give credit to it, so I’m going to draw quite a few things 
from it — it * s called D ynamics of Change . Published by the Kaiser 
Aluminium Company «. And now one of the things I - want to do is read the 
introductory statement to you. I thought it was kind of interesting. 

*At exactly 0513 on the 18th of April, 1906, a cow was standing at 
liy - 20‘ West Longitude, 37* - 58’ North Latitude. Somewhere between 
the main barn and the milking shed on the old Shafter Ranch in 
California. Minding our own business, suddenly the earth shook, the 
skys trembled, and when it was all over, there was nothing showing of 
the cow above ground but a bit of her tail sticking up. For the 
student of change, the Shafter cow was sort of a symbol of our time. 

She stood quietly enough, thinking such gentle thoughts as cows are 
likely to have, while huge forces outside her pen built up around her, 
and within a minute discharged it all at onee in a great movement that 
changed the configuration of the earth, destroyed a city and swallowed 
a cow. If we do not learn to understand &nd guide the great forces of 
change at work on our world today, we may find ourselves like the 
Shafter cow, swallowed by vast upheavels in our way of life quite early, 
somewhat.’ And these kinds cf changes, I think we all recognize these 
kinds of upheavels in this world are going on now at a rate magnitude 
of which is completely beyond our understanding anything we thought 
possible. This of thinkers commenting in various ways on this. I 
noticed in his publication, said that we’re living through the second 
big device in human history, comparable to the magnitude only to that 
first break in historic continuty from barbarism to civilization. 



Nature and magnitude of changes are surprising, and here come a 
few of those statistics, that I mentioned to you. 25% of all the 
people who have ever lived are living now — that’s an astonishing thing 
when you stop and think about it. Ninety per cent of all the scientists 
who have lived are living today. Half of all the energy consumed by 
man has been consumed in the last 100 years. And I was reading in a 
medical journal the other day that medical knowledge has advanced more 
in the last sixteen months than it has in all history. My own experience 
of late is when I was in high school — nuclear energy and automation 
hadn’t even heard these words. They weren’t even part of our vocabulary 
at that time. So clearly technicological breakthroughs in the last 
twenty years »iave a tremendous effect on what’s happening now and what s 
going to happen. Perhaps one of the most significant things though is 
the changing nature of the Earth in terms of the crush of population. 

This is what everyone’s talking about. You’re all familiar with the 
figures, even then they shake you when you look at them. From the 
beginning of time to 1850 we reached the population of a billion. We 
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doubled this and got 2 billion in the next 75 years and we doubled 
again and we f ll get 12 billion in the next 50 years— 1975— if you - re 
familiar with the geometric progression. One of the interesting things 
here I found the other day while I was doing a little reading in this 
area. Actually most of this doesn’t have particularly much to do with 
the birth rate. The publication I was reading ascribed to DDT, 
penicillin, and soap, as the three most important things in recent 
history in increasing the population of the world. In Ceylon, for 
example, DDT alone was responsible for increasing the population by 
83% in one decade. Corresponding decrease in per capita income — 
corresponding decrease in standards of people. Even when you think 
about this geometric progression and understand it, the topographers 
tell you that 600 years from now at out present rate of population 
growth, there’ll be one person on every square foot on earth. It’s 
surprising a little bit — you know it won’t happen. Human beings and 
arithmetic are two entirely different things. We’re going to reaqh 
that point. We’re trying to figure out what we’re going to do in the 
meantime. This adds a new dimension to the kinds of changes that are 
going on now. Of course, this would all be different if people were 
spread evenly across the earth, but being gregarious, individuals 
tend tc gather and populate the places. One significance of this 
from the standpoint of change, I’d never thought about, reading an 
article in the area of urban ecology, talked about colonial times when 
we had a population of one per square mile. Inside a twenty mile 
diameter circle, because the figure 10 miles was about as far as 
anyone would walk to work, to the fields and back — about 334 people. 

The chance of contact was about 333 to 1; not too great. Chicago has 
a population density of 10,000 per square mile so in the same circle 
you have 200,000 people. Chances of human contact are 199,999 tc 1 
instead of 333 to 1. People come into contact with one another, new 
ideas develop, new problems develop, things change. You get more andr 
more people coming into more and more close proximity with each other 
and things are changing faster. 

Well, about now, I will imagine you’ll be thinking what does all 
this have to do with budgeting; well, I’m not absolutely sure. Here 
are some of the things, I think they may mean that have some relevance 
to budgeting in a very broad sense. How do we use public money to do 
the things we want to do? Technology has reached a point where we can 
increase production at any level we wish by greater investment in 
production. Our problem is not to increase production, but to create 
the kind of society that can enjoy the new technological world we’ve 
created. Objectives of government and business are becoming much more 
closely related than ever before. This is a difficult proposition in 
a way because all related business has to do scarcity. It was always 
the rationing of scarce resources. Now we’re faced in many areas with 
overabundance. This broadens our choices and alternatives. Making 
decisions about what we’re going to do has become more difficult. 

Another area closely related, as far as creation and utilization of 
resources is concerned, is the truth. The central truth about the developed 



countries today is that they can have anything in the shortest run, the 
kind and scale of resources they decide to have. It's no longer 
resources that limit decisions. It's decisions that make the resources. 
This is the most fundamental revolutionary change mankind has ever know. 

I don't know what is the most significant change mankind* 8 ever known 
but it certainly is a very significant one. Another one, and I talked 
about this briefly, is the availability of public money. We know that 
population and rate of economic growth will produce instrumental 
increases in public money beyond anything we could visualize before. 

I mentioned an incremental $10-15,000,000,000 per year Increase in 
Federal resources, and this is not withstanding Viet Nam or anything else; 
we can have small wars. I’m not sure we could have big ones. This makes 
the decision more difficult. 

Certainly another significant one — we* re seeing this more and more 
every day is the orientation of our society toward youth. Not only 
in numbers but in terms of attitude. I thought this was interesting 
when I saw this particular statistic — that by 1986, half the people in 
the world will be under 15 years old. And the children in China under 
ten are greater than the entire population of Russia. I'm not sure what 
these things mean, but they surely must mean something in terras of the 
kinds of changes that are going to be occuring in our attitudes. 

Certainly one of them is that now as cur median age is going down more 
rapidly in the last years than it ever has before in our history. Now 
the majority of our people were born after the Depression of the *30' s'. 
They were born by and large in a period of and their attitude toward 
spending government money, are tremendously different from those who 
were born, or who were raised or who's ideas were shaped during the 
Great Depression. Then the whole idea was never spend a dollar today 
because you may need it tomorrow or put it away for a rainy day’ — this 
sort of attitude. Governmental attitudes toward government spending are 
hanging in the balance now and they're moving over to an entirely 
different concept. People who are making the decision about them are 
different people~--entirely differently conditioned. 

Certainly another major area of significance is the confrontation 
that is occuring and is going to keep on between the haves and the 
have nots. Technological advances, abundance of global communications 
are obvious for all of us to see. And certainly this type of confron- 
tation is going to have a tremendous impact on how we operate in this 
country and all over the world, we're saying here on the streets of 
Washington, Lcs Angeles, Chicago, and every place else, an education is 
certainly what we're interested in. H. G. Wells once said that human 
history more and more becomes a race between education and catastrophy. 
That was 30 years ago. Even though we recognize the truth in this 
statement, we appear to be struck to what we're going to do. I could 
quote John Wilkenson, who wrote "...Society" He said, "One information 
processing device that cannot easily accomodate itself to acceleration 



of change, is the human brain. " Generals are always fighting the last 
war, educators are always instructing the last generation. The average 
college student is a very badly programmed computer. This is an 
interesting comment because I think of it in connection with BBS — this 
is a kind of change we're experiencing here and we find the kind of 
resistance that occurs. 

So much for change, just as well not dwell on them too long, we 
might if we did include that it's hopeless that human beings just 
aren't constituted in a way that they can adjust to kinds of changes 
they're going to have to adjust to. I suppose that regardless of all 
these things, least those of government, we don't have any real option 
but to make the attempt. We can take refuge in the fact that no less 
a person George Bernard Shaw said, 'Government always makes the right 
decisions, but only after exhausting all other alternatives.' In PPB 
we're trying to think in terms of looking at the other alternatives 
before you've exhausted them and not arriving at the decision only 
because it's the only alternative left. In budgeting I'm not sure 
we've exhausted all the alternatives because we haven't really tried 
very many. As a matter of fact, most jurisdictions in this country 
prepare budgets in just about the same manner as they did in ancient 
Rome. 



Let's take a brief look at what budgeting is, or at least what it 
ought to be. First of all, it ought to be a translation of program 
objectives into dollar terms. And a statement of the resources required 
to meet those objectives. Second, it ought to be a statement of major 
public policy — in a society of limited resources, we must make a 
choice — and the budget is the way we express these choices. Third, it 
should be an expression of the fiscal policy of the government — the 
total volume of income and expenditures. Some of this ought to be 
pretty elementary in terms of budget preparation and administration — 
I'll pass over it very quickly — they're some degrees of significance 
to a few of these items. You know the process divided into four parts, 
formulation by the executive, usually at least, authorization of the 
legislature, execution by the executive, and then accountability, which 
may be either an executive or legislative function. Actually, these 
distinctions are a little bit cloudy. Not so much because they aren't 
clear distinctions, but because there's a tendency between executive 
and the legislature to shift to one of these areas to the other as a 
means of getting off the hook with a difficult kind of decision. 
Legislatures will in a particular area, sort of back off; the executive 
authorizes this because it's a very unpopular decision and we simply 
don't want to make it. In the terms of execution, the legislative 
body may try to direct his administration of something to influence the 
way the executive executes. But by and large, these kinds of 
distinctions do hold up. 

Actually, I suppose in this country we can date what we call modern 
budgetary practice back to the Federal Budget and Accounting Act of 
1921, imposed certain kinds of criteria on the development and presen- 
tation of the budget and most of the states about that time sort of fell 
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into line following the same kinds of practices. At that time up 
until 1950, all budgets were line item* Most of them still are. 

That is, they consist of a series of objects of expenditures, personnel 
supplies, equipment and so forth. They focus not on what government 
wishes to accomplish, but upon the goods and services it wishes to 
acquire. Line item budgets are heavily orientated toward 
expenditures control, are usually prepared and administred by 
accountants. Around 1950, a few more progressive jurisdictions 
started trying to build a better budgetary mouse trap and moved into 
what has been called performance budgets. Performance budgeting is 
orientated toward the cost of activities rather than objects. It 
leans heavily on work leading to see cost approaches. It still dwells 
on short term costs — involves analysis of total costs, but give less 
weight to control in the line item budget f which is usually prepared 
and administered by a person with some analytical skills. In both the 
line item budget and performance budget — and Oregon has had for 12-14 
years a fairly sophisticated performance budget. In both the line 
item and the performance type operations, the budget development 
always starts at the bottom of the organization the lowest organizational 
unit develops what it considers its needs and this moves on up in the 
organization. That goes on and you finally get to the top and the total 
budget is quite the opposite of program budgeting. Late in the f 50 ! s 
some government started to move toward what has been termed programmed 
budgeting, although there f s a great deal of confusion in the use of 
this term. Quite frequently it has been applied to budgets that were 
programmatic in format only. The budget was divided up into program 
divisions, but it was really a line item budget or perhaps a performance 
budget. 

A true program budget depicts the total cost in both the short 
and long term of achieving predetermined programmed objectives. The 
budget is mission oriented. It’s highest development leans heavily on 
the cost effectiveness analysis. Work load and unit cost are 
frequently employed, but only in terms of measuring dollar requirements — 
not in terms of analytical decision making. Budget planning starts 
at the top organization in the unit and proceeds downward rather than 
from the bottom up as in the case of the other approach. And usually 
programming budgeting requires persons with management and systems 
analysis types of skills* Many jurisdictions are trying to weld 
planning program and budgeting into a system which can aid decison 
makers in analyzing alternatives, establishing priorities, and deal with 
the objectives of governmental programs as a unified whole rather than 
uncoordinated segments. Here in our state, not much different from 
others, in reviewing the budgetary decisions the Governor's office followed 
two basic criteria, one relates to the end — how important is it to 
accomplish the magnitude of the service you want to perform, and the 
other relates to means, the work being carried out now? Sometimes, 
there has been too much of .a tendency to look at the means rather than 
than the end. Examining whether this is really scope in magnitude of 
services you ought to be performing. Relating to levels of service, some 
kinds of guide lines are these: First, one of the questions that I should 
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say we would apply to these decisions, what's the objective, and what 
are the alternative ways of achieving the objective. How was the 
existing level of service originally established, what are the 
relative priorities, what's the public policy tradition, and what's 
legislative dispositor. toward the particular item because the politics 
of the situation are very real. At this point, a budget would be 
changed one way or another because they didn't fit our tradition or 
didn't fit the political realities of the present situation. In terms 
of these guide lines related to the efficiency of work done, we apply 
a great deal of analysis in the area of work clothes, in the area of 
standards, in the area of development of unit cost, in the area of 
management improvement, such as work simplification, technical improv- 
ments, etc. We will look at budgets in the terms of what overlapping 
may exist between related functions, duplication, etc. 

Major problems in our budgeting effort, and I suppose in most 
other jurisdictions, is lack of definite program objectives. We don't 
really know what it is we're trying to make. Objectives were really 
activities, and what we were trying to achieve was pretty much what we 
had always been doing, what we had always been doing only let's try 
to do it better if we can. Lack of planning, lack of consideration of 
total cost in making budgetary decision, making only immediate costs 
for one budgetary period, lack of top management capabilities and that's 
still a problem and will be for quite a period of time. Lack of 
vigorous executive advocacy of programs. Acceptance of existing level, 
incremental budgeting; you don't ever look at what you're doing now, 
you only look at those little incremental Increases that each year you 
add on top and finally achieve a double standard in expenditure of 
public funds. I'm sure most every state has some amount of money 
derived from certain kinds of revenue, usually called th;* general fund 
or some language of this kind. It's available for any agency that can 
obtain it. Then you have your dedicated funds — those moneys that can 
only be spent for specific purposes — where there is no competition. 

In o.rlier words, you say that highway taxes are available only to build 
highways, as an example of the non-competitive fund and the general 
fund and you get the standard that exists between two broad segments of 
state expenditures. 

With that review we might talk about a fee problems that relate 
specif i;. ally to education at least as we see them in this particular 
state. By any measure, education is the most difficult function in 
which budgeting experts have to deal with. Among the problems encountered 
in Oregon as we attempt to bring a greater meaning to the process of 
budgeting for education are these: First of all, fragmented organization, 

traditional division of functions between state and local governments. 
Rigid segmentation: Two-year post high school, four-year post high 

school, graduate education, continuing ediication, etc. Further 
restrictions within each segment. For example, secondary schools, general 
education, special education, vocational education. We tried various 
coordinating kinds of mechanisms; they haven't worked by and large. This 
is a real problem to us, and this is one of the areas, in terms of 



bringing together related programs in terms of common objectives that 
PPB can be meaningful. It can be more meaningful in a state like Oregon 
that has a highly fragmented overs tructure, than it might be in some 
other jurisdiction. Another factor is the inability to deal with 
education as a proposition resisting a notion as Professor Neal Chamberlain 
of Yale said, f that a person can acquire in the first twenty or so years 
of his life, all the formal education he needs to keep him on an 
ascending career through the remaining forty years of his working life. 1 
I think that the things that I talked about in terras of the kind of 
changes that are occurring throughout the world indicate a fallacy of 
this — that education is growing and must be a continuing process. But 
in this area we’ve made very little progress in sorcing out the relative 
responsibilities of the various communities, non-educational, governmental 
agencies, business industries, and other elements of the private sector. 
Certainly, this isn f t something that can be- laid right at the door of 
organized educations It f s something that involves the total of our 
society. 

Another area, is the inability at least thus far, to develop the 
program structure for education that adequately depicts its functions. 

It can f t properly relate educational programs to each other and other 
governmental operations unless we understand the nature of this structure. 
In higher education in Oregon for example, we have a program called 
IRRGS (instruction, related research, and general service) . This program 
serves some 55,000 students, consists of hundreds of sub-programs, and 
is so broad as to be totally meaningless in terms of planning and budget. 

Of course there is lack of clear educational objectives. None of the 
efforts have been to try to develop objectives for all state programs. 
Incidentally, in that area I have been able to bring three booklets 
here — a working draft of our program and its objective for this state. 

In this attempt to develop objective in the area of education, our efforts 
just haven't worked out too well. They tend to be activity rather than 
objective orientated and are frequently so loaded with abstractions such 
as quality and excellence, that they 1 re not susceptible to modification. 
When we want to improve the quality of education all you can say to that 
is that it's very interesting. But what does it mean? 

Another problem area is lack of a relevant information system. 

Modern planning and budgeting is powered by information. Education 
produces vast quant itites of information, much of which is simply not 
relevant to a decision making problem. It’s never been systematized 
In terms of what is the specific kind of data we have to have to make 
specific kinds of decisions. It f s just a vast accumulation of garbage 
for years and years — we've always put these things together so we let 
each legislature ask for some more and every Covernor ask for some more 
and it gets superimposed on the existing amounts. It f s just not relevant 
to what we need. 

Another problem area is the lack of relevant output indicators and 
valid measures of objectives of activities required. Worse than the 
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lack of these things, is the presence of a number of superficial things 
which tend to give a plausibility to some rather implausible things. 

This in education, in higher education, is the student teacher ratio, 
which has a limited usefulness; but has been extended far beyond the 
limits of its utility in terms of using it as a decision making tool 
expenditures for education. 

A seventh area, a continuation of obsolete functions. This is cert- 
ainly not to single out education. It occurs in many areas of government, 
usually manifested by a program gradually shifting its objective to 
areas where its continued existance can be justified. I can think 
in this state of two areas of higher education that qualify for this 
kind of description. One’s continuing education in which the develop- 
ment of community colleges throughout the state has greatly changed the 
requirements and direction of continuing education. By its continued 
existance, continuing education has started to move into other areas. 

Such as sending teams around the state to conduct seminars on civil 
defense or move into other kinds of areas where it will somehow justify 
their continuation. Federal extension is notorious in this area, 
geared ages ago, to a rural economy which has since changed and is 
trying desperately now to move into programs for the disadvantaged, 
programs geared to urban communities as a means of continued existence 
even though it may be competing vigorously with a dozen established ag- 
encies in those areas where it operates. 

Another problem area is inability or unwillingness to analyze 
alternatives and weight priorities. This occurs not only between 
education and other functions, but between segments of education. For 
example, this is one that bothers me, I hope that someday we can devote 
some real hard analysis to it. Oregon exports graduate professional 
students. We don’t use them in this state to the degree that we could 
we export them to other states. On the other hand, in terms of 
vocational and technical f lining, we just about hold our own, not very 
much better. Yet we spenu proportionately a great deal more for 
graduate professional education than we spend for vocational— technical 
education. There are a number of implications — some broad and some 
narrow. What is Oregon's parochial self-interest as compared to making 
a contribution to the national good by producing educated people to send 
other places? What’s the kind of social, economical climate we want 
to create in this state? What is the cost effectiveness, and marginal 
benefits from shifting resources either toward vocational-technical 
education or further toward graduate— professional education? We simply 
have not faced up for the need to do hard analysis and make hard 
decisions about these questions at this time. Not only in education but 
in other areas. But in education, particularly, and the example I gave 
is a pronounced problem." 

Attention turned to planning strategies for implementation and 
"picking up the loose ends" which become apparent in a two-week Institute. 
Dr. Allen Lee, Director, Teaching Research Division, OSSHE, led the 
discussion and instructed on the topic, "Planning to Implement PPBS 
at Horae." The basic instruction was on Program Review and Evaluation. 
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Techniques (PERT) as a technique for charting the tasks involved. The 
basic format • included deatiling the activity, listing personnel 
assignments , estimating man days required, estimating beginning dates 
for each activity, actual beginning date, target completion date, and 
actual- completion date. With much help on detailing the activities, 
participants were assigned the task of PERTING imp lenient at ion tasks 
for their home situation. A sample of the PERT charts utilized follows 
on the next page. 
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PPBS PERT CHART* 
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Friday, August 30. 

The first session was spent in individual and buzz group reports 
on the PERTING assignments given Thursday. These showed that quite 
thoughtful, detailed plans had been formulated by the Institute 
participants for PPBS activity on the home front. Dr. Lee answered 
several questions on the Perting technique. Dr. Lee also led a 
discussion on next steps needed in the PPBS movement, both fit the federal 
and state levels. 



The final major presentation was given by the Institute Director 
and was designed to pick up loose ends and reply to the several problems 
which the skeptics had raised during the Institute. His talk was 
entitled, "Skepticism and Limitations of PPBS." 

Dr. McAbee introduced topic by stating that there is usually 
a need to assume an objective head on some of the issues involved in 
PPBS and provide answers for the skeptics. This is a common necessity 
in every new movement. 

"One of the common types of skeptical views heard goes something 
like this . "It * s the same old thing in a new package . * If this is your 
posture you are using the ploy as an excuse not to change anything * 

Or. you may be ahead of the game and are already doing PPBS . Or perhaps 
you’ve already been able to fit all your preconceived prejudices into 
a new pattern. My reply to these viewpoints is this. PPB is really in 
the developmental stage and moving toward its maximum potential. There 
is every opportunity for you to shape, adapt, and improve it. In 
so doing, you will be making a general contribution as well as improving 
your competencies in your own job. 

A second major skeptical question raises the question o^f lack of 
personnel to do the new job. This plaint is that doing the new thing is 
therefore impossible. Now, there is some truth to this posture, but 
in my opinion, it is greatly exaggerated. I believe the answer is to 

learn to use PPBS rather than whatever other system is in use. It takes 

a different know how, but will probably save work in the long run. 

Another complaint abou‘; PPBS is that it tends to push major 
decision making higher in the organization and concentrates authority. 

This complaint probably stems from the use made of PPB by former Secretary 
of the DOD, Robert McNamara. One very realistic answer to this question 
i 3 that, in my opinion, the trend is to move essential decision making 
upwards. This trend seems to develop when the amounts of money increase 
greatly and the programs become controversial for any reason. The 
locus of decision flows where the heat and the money are! Alright, why 

fight it; the big boys should earn their salaries anyway. Another 

personal opinion, born of some experience and reading in the field, is 
that the level of essential decision making will be concentrated or 
delegated according to the propensities of the higher echelon personnel. 



Whether the management system is PPBS,"hit or miss," or traditionally 
organized, the top people can, will, and do decide where tie ec s ons 
are made. The criticism is not relegated only to PPBS; it depends on 
how the system is used. X would be relatively certain that a goo 
job will be accepted and thusly, lower echelon personnel will come to 
have more and more voice in decison making. 



Another noint of skepticism. 'Its all political anyway, why bother 
with all the work in PPBS . 1 I think this is a 'yeah, but--- kind of 
answer . I'm not ready to deprecate and discard a logical, factual 
basis for decision making. If one thinks the process is useless, then 
we'd better start looking for another form of government. I m not ready 
to do that. I believe that most of you feel that we can positively 
influence the politicians who make the decisions. I, for one, believe 
we can make a more effective case for education with PPB systems. 



Another skeptical point. 'PPB takes management and control out 
of budget making and operation.' There are always the three aspects 
of the process: planning, management, and control. PPBS emphasizes 

planning and evaluation to improve management and more accurately, xts 
effectiveness. Control becomes program centered where it should be. 



Another plaintive cry is, 'you can't program education.' I b ey® ve 
this one comes from those who know so little that they are afraid that 
PPBS turns the operation over to the computers. 

It is difficult, but it is possible to systematize and program the 
management of the educational enterprise. Our critics will note the 
advance and applaud. We can specify our objectives and measure the 
results. At least, a start can be made; we can t get caught stating 
that it can’t be done. As effort is expended we will refine our 
efforts and develop new techniques* 



'It can't be cone in our situation* is another plaintive cry 
frequently heard as rationale for not doing anything new. In certain 
that PPB will not escape this charge. It. is probably the oldes negativ 
posture known to civilized man. Let us take a positive stand and state 
that the system needs much adaptation, but that we will make every effort 
to accomplish something with it. 

Have you heard, 'We tried PPB and it didn't work' or 'It sounds good 
in theory, but won't work in practice'? I submit that the state o t *e 
art (PPBS) is such that it couldn’t have had much of a trial in educati n 
to date. If the theory is sound, the practice will be good x w 
inept in its application. Of course, the rationale for the Institute was 
to learn enough theory and application to give PPBS a xair trial. 

As in most every new movement, problems and implications will present 
themselves at some future time which cannot be forseen. ese wx 
to new opportunities for solutions and development. 
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A parting word. Your participation in this Institute has been 
most appreciated and worthwhile from my point of view. Thank you all 
very much for your wholehearted cooperation and substantial effort. 
The very best to each of you in your future endeavors!" 




PART IV: INSTITUTE EVALUATION 1 



No single evaluation model will adequately suffice for the range 
of decisions that must be made during an on-going institute. A focus 
on student behaviors, on management decisions, or on broader educational 
needs requires that special questions be asked. Different foci require 
different bodies of information and varying modes of interpreting this 
information. What information to gather, what audience to serve, what 
language to use are all difficult pre-evaluation decisions. 

The instruments designed and applied in this institute were 
addressed to four basic types of evaluation. Specifically, these were 
context evaluation, which serves to define the environment; input 
evaluation, which enables decisions on necessary outside assistance, 
appropriate strategies, and the design or procedure for the selected 
strategy; process evaluation, through which defects in procedural design 
or its implementation are either detected or predicted; and product 
evaluation, which attempts to determine the effectiveness of the 
institute. This last form of evaluation attempts to relate outcomes to 
the stated objectives. 

Evaluation of this institute met an initial hurdle in the objectives, 
which were net stated in such a manner as to be readily operationalized. 

An effective and interpretable evaluation is dependent upon an hierarchy 
of objectives, stated in behavioral or operational terms. The objectives 
for this institute could have served adequately as goals or directives 
for the instructional and administrative staff but were difficult to 
interpret in an evaluative sense. For example, such terminology as "To 
acquaint", "To develop need". "To thoroughly orient," is virtually 
impossible to define against some overt, measurable criterion. 

Use of a pre- and post-measurement technique was both valid and 
unique in evaluating an institute of this nature . (see Appendix A) . 

As may be noted in axtachment A, however, the statistics from this test 
were too inconclusive to be of any decision value. Lack of any signi- 
ficant differences in statistics calculated indicated that no further 
statistical tests were either implied or acceptable. The calculated 
gain score was . 31 , which also indicated that either the evaluation was 
poor or the instructional program was weak.^ A more sophisticated and 
discriminating evaluation program is necessary to determine which element 
is liable to modification or revision. Despite the shortcomings of this 
instrument, there is little doubt about the value of this type of an 
evaluation effort and a valuable post-institute study would be the 



■^-Editor's note: Part IV was prepared by Dr. Frank Nelson, Assistant 

Research Professor, Evaluation Unit, Teaching Research Division OSSHE. 

2 

The test was not designed to measure growth gains during the Institute, 
but only to focus attention on the overall task. 
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development of an instrument that would satisfactorily accomplish the 
intended evaluation outcome. This instrument could be used for future 
institutes of a similar nature. 

The participants in this institute were apparently a very diverse 
group, both in terms of background and academic abilities. Some 
pre-assessment of who they were and what experiences they had undergone 
would have been valuable. Due to the variance evident in experience 
and abilities, perhaps some type of common denominator should have been 
established for building the cohesiveness of the group. For example, 
persons with some background in PPB could have been used as small 
group leaders which would have provided direction for the small group 
study and developmental efforts, while concomitantly prof icing these 
persons with an opportunity to add constructively to the institute as 
a whole. 

It was also apparent from comments of the participants that a good 
many more examples of on-going PPB efforts would have been valuable. 

These may have been either existing programs, examples generated by 
participants, or models developed by the instructional staff. Unfor- 
tunately, in relation to precise interpretation, the polar counterpart 
of every statement was present. 

Some concern was expressed for the lack of REAL flexibility in the 
institute schedule. While most of the participants were commendatory 
in their remarks about the sensitivity of the staff, there appeared 
to be several instances where consultants needed additional time. It 
also appeared that most of the participants would have welcomed a 
greater opportunity to prepare for the institute in advance. Mailing 
institute programs or agendas, reading lists, and other types of materials 
with which they needed to become familiar may have been a valuable 
strategy. 

Generally, the following observations would seem appropriate: 

1) There was a need for a programatic evaluation policy for 
monitoring and adjusting the institute emphasis to the pace and needs 
of the participants. 

2) An apparent positive shift in attitude toward PPB was evident 
in the comments made by the participants. 

3) Pre-ins true tional groundwork needs to be laid more carefully, 
i.e., pre-assessment of the participants. 

4) This evaluation suggests that ALL critical elements of the 
institute be identified and weighed, each in relation to the others. 

The important elements, in terms of key constructs of the institute, are 
then identified and can be carefully considered in necessary decision 
making . 
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5) Greater budgetary allowances must be made for evaluation if a 
satisfactory third party evaluation is to be conducted. Addi tonally, 
the evaluation team must be involved in the total planning and imple- 
mentation of the institute* Only then will such an evaluation be of 
any meaningful value. The evaluation program executed for this institute 
is incapable of determining either which aspects of the institute 
require modification or revision, or what behavioral changes were 
manifest in the participants. 
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ATTACHMENT A 



Ind. 


Pre 




x i 


Post 


x 2 


x 2 

2 


A 


4 


-2.35 


5.52 


8 


1.5 


2.25 


B 


4 


-2.35 


5.52 


7 


-.5 


.25 


C 


5 


-1.35 


1.82 


6 


-1.5 


2.25 


D 


5 


-1.35 


1.82 


6 


-1.5 


2.25 


E 


5 


-1.35 


1.82 


4 


-3.5 


12.25 


F 


6 


-.35 


.12 


9 


2.5 


6.25 


G 


6 


-.35 


.12 


5 


-2.5 


6.25 


H 


6 


-.35 


.12 


9 


2.5 


6.25 


I 


7 


1.35 


1.82 


7 


-.5 


.25 


J 


8 


2.35 


5.52 


9 


2.5 


6.25 


K 


8 


2.35 


5.52 


8 


1.5 


2.25 


L 


9 


3.35 


11.22 


10 


3.5 


12.25 


M 


9 


3.35 


11.22 


10 


3.5 


12.25 


N 


9 


3.35 


11.22 


7 


3.5 


12.25 



Sum 


89 


63.38 


105 


72.50 


X 


6.35 




7.5 




S 




4.87 




5.58 


s 2 




23.72 




31.14 


Xmdn 




6.33 




7.5 


Gain score 


= 16 


= .31 







140-89 
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ATTACHMENT B 



(1) Was the Institute staff responsive to the needs and probl.ems of 
the participants? (Give one positive and one negative example) 

fWsitiVcr Negative 



3 .'.ions - last 

tjere vocational 
f'/iacril vi* tjrlstnted. 

ProgiT^Ta v *>5 adjusted to meet 
the needs cc the participants. 

Repeat material at different 
levels of consideration. 

Staff was most cooperative.. 

Many opportunities for questions, 
answers and discussion. 

In the cost effectiveness and 
cost benefit analysis 
applications . 

Flexibility in scheduling 

Orientation sessions well geared 
to entering knowledge of the 
participants. 

Good job of covering all levels 
of PPBS and skillful scheduling. 

Good (wide) range of resource 
people. 

Generally satisfactory. 

Understanding and purpose of 
PPB was well covered and 
informative. 



Questions from the participants 
were entertained at all times. - 55 - 



Not enough content for 2 weeks - 
set faster pace. 

Group sessions not well structured 

Program case studies could have 
been more relevant to vocational 
education. 

Not enough detail in ONE case study 



Needed more direct application to 
vocational technical education by 
use of PPBS. 

Needed more case studies. 



Two persons who dealt with out-of- 
state problems were of little value 



Some provision should have been 
made for those who had prior 
, experience with PPBS - they caused 
some problems in requiring an 
abnormal amount of instructors 
time. 



(2) What activities were most, productive for you and which experiences 
were least productive, e.g., large presentations, small training group 
work, panel work, individual study work, etc. 



Productive 

Small group - last problem. 

Case study from Ohio. 
Presentations on analysis. 

Small training group. 

Problem solving in small groups. 

Group work. 

Small groups - especially in 
working out specific problems. 

Case study - p.m. of the 27th; 
group work; some presentations. 

Small training groups. 

Small training groups, especially. 
Small groups. 

Small groups. 

Small groups. 



Non-productive 

Group reports were drags - poorly 
done - especially for educators. 

Some of the general sessions. 

Panel work* 

Individual study work. 

Several of the large presentations 
were superflous. 

Some presentations could have been 
shorter without sacrificing 
content . 

Individual study work - too many 
redundant references c 

Lack of annotations on the bibli- 
individual study work; some 
presentations . 

Panel work. 

Large presentations (objectives). 
Individual study groups. 

Large .presentations . 

Large presentations (some, not all) 



(3) In this Institute, we were concerned with giving information, 
providing for the planning of implementation for PPBS, and for the 
development of a set of guidelines for applying such a PPB system in your 
state. Would you have preferred more or less experiences in each of these 
areas? 



Comments 

(a) Giving information: 

“I am well aware of the concept.” 




Question 3(a) continued: 



Used an abnormal portion of time in identifying problems and continuous 
objective writing. 

Lack of identifying those who had some background in the area. 

Shorter period on general information and more on specific. 

Need an hour on appropriate economic principles and definitions. 

Vocational education presentations on the first day should have been 
delayed until second week. 

Appeared adequate. 

^"Elt that the general information provided was good. 

Background information on the subject was available before we can go to 
the institute. 



(b) Planning for implementation of PPBS: 

Espe' tally staff training. 

Little time devoted to strategy for implementation. 

"Since this is the first stage, and since most states won't be beyond 
this point for X time, perhaps the coordination of timing could have had 
more attention.” 

Maybe a whole day on this. 

More group work on planning specific problems. 

Practical situations/problems to work on. 

Perhaps through small group work study groups this could be achieved. 
More from the standpoint of a state director of vocational education. 



(c) Development of guidelines for applying PPBS: 
Group was able and ready to move faster. 
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Question 3(c) continued: 



Need a program structure for vocational education. 

More discussion of setting calendar and activity requirements. 

*'X suppose everyone would like a more specific guide on tiow to go 
home with A-B-C steps (probably not feasible)." 

Planning and development would let us take home a plan for action on 
paper. 

Enough given. 

Maybe several forms could have been developed for some specific 
programs . 

(4) Generally speaking were our instructional objectives relevant to 
your needs in the field? 

Most relevant Least relevant 



What PPBS is and problem development. Whay are your problems? 
Planning for implementation. 



Implementation of a PPBS system. 



Political realities of the 
budget making process. 



Sessions on analysis. 



Session objectives. 



Need case problems in vocational- 
technology. 



All three. 



Processes . 



Small group activities applying 
PPBS skillls. 



First day agenda. 



Being a novice at PPBS most were 
favorable. 



Out of state probJ.eas 



Understanding of PPBS and 
strengths of the system and how 
it can be of value. 



Detailed study of cost, etc. 
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(5) What instructional objectives should be added to make the Institute 
more relevant? 

Aware of sources of data on vocational education. 

Pre-planning - information disseminated to participants, e.g. agenda, 
conditions . 

Possible individual work. 

Give participants a model implementation plan. 

The generation of more case problems similar to one we attempted to 
analyze from the Ohio State Center. 

“If we are interested in developing guidelines for implementation we 
would structure the learning experience to work with this aspect.” 

Development of more case studies. 

Evaluate projected activities report in terms of PPBS. 

Small groups should be better organized - objectives should be clearly 
stated for group sessions. 

More realistic problems, consultants, and a computer. 



(6) What effects do you think this Institute will have on the perfor- 
mance of your job next year? 

Inspired to learn more about application of the system. 

A better program budget will be developed. 

Able to take further steps in the implementation of PPBS. 

Be more alert to the establishment and review of objectives. 

"Will make it more difficult. A staff in-service training program will 
be written and implemented." 

Back-up data for our budget will contain more analyses and consideration 
of analyses. 

Believe 1 will be able to make a more objective presentation of vocational 
education needs to budget people. 

Tremendous improvement. Has increased my confidence to do my assigned 
job: get PPBS implemented. 



O 
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Depends on progress of other agencies. 

Greater understanding of the role of analysis in the process* 

I am in a position to more intelligently pursue PPBS analysis in my 
own program. It is to be understood that supervision is still needed 
with addi tonal instruction. 

More confidence - at least in terms of terminology. 

As programs are suggested 1*11 be able to ask for information which 
will assist me in decison making. We *11 use PPB to the extent possible 
with staff available. 

Need not wait for next year - the job needs to be done immediately. I 
will review what I have not in draft form and revise it accordingly. 



(71 Please give a positive and negative comment on each of the following 
Institute activities. 



(a) Large group presentation: 
Positive 

Information not available in the 
literature was presented — 
i.e., persons experience. 

Speakers well informed. 

Interchange and dissemination of 
ideas and information. 

Competent speakers. 

Some speakers, through their 
experience had much to share. 



Helped to bring resources of the 
group to bear on important points. 
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Negative 

Not necessary to identify problems 
need to discuss solutions to 
* problems . 

Need to use more effective 
presentation strategies 

Group too large and 
heterogeneous . 

Discussion of personal experiences 
(only) is of limited value. 

Appeared that speakers never had 
enough time. 

Too much pressure to maintain 
schedule many speakers had to 
quit when the discussion just got 
going. Some speakers did not 
have an orientation to the group 
and its interests. 

Too many people have to relate 
their total experiential back- 
ground in the field* 



Question 7(a) continued: 



Gained much information. 



Some were very bad. 

(b) Training qroup activities: 
Positive 

Ohio case problem very good. 



Excellent way to practice the 
process being learned. 



Well spaced through the list. 



Good opportunity to apply PPB 
skills which were presented. 

Grouping more flexible and 
beneficial. 

Enabled participation on 
specifics. 



Re-examination of materials 
presented . 

Speakers (some) wander too far 
from the material under 
consideration . 

One or two presentations were 
excellent. 



Negative 

Need help on reporting procedures- 
secretarial help, machines, 
better teaching materials (av) . 

Not well structured enough. 

Case studies subject matter seemed 
to detract from objectives. 

Few members desired to force their 
pbint of view. 

Need more simulated activities 
whereby group works along with 
leader in large group before 
going into training group. 

Needed time to produce presen- 
tation materials to increase 
yield and stimulate discussion. 



In certain areas a group leader 
from the staff could have 
stimulated further discussion. 

Could have used trained leaders. 
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Question 7(b) continued: 



Excellent exchange of ideas. 



(c) Panel discussion: 

Positive 

Differing view points valuable. 

Good reactions from members 

Exposure to diverse outside value 
judgments. 

Use of politicans and state 
personnel gave an excellent view- 
point on dealing with legislators 
and budget personnel. 

Pollowup needed. 

Stimulating, and helps to bring 
into focus what we have and have 
not learned. 

Involved more people. 

Kept on subject. 

Brought in a variety of ideas. 



Group sized should have been 
controlled better - participants 
should have been rotated to 
induce better exchange. 



Not enough time alloted. 



Negative 
Repetitious . 



Not enough time. 



Participants be given more time 
for prior preparation - 
schedule for a larger block of 
time. 



Some panel members appeared to 
lack a depth of understanding of 
PPB. 

Panelists were not prepared in 
advance . 

Generally poor - seemed to be 
structured at the last moment. 
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Negative 



(d) Small group activities: 
Positive 

Should have at least 5-6 
persons in each group. 

Problems were to the point. 



Excellent means of developing 
ideas together. 

Good in most cases. 



More individual effort resulted. 
Good mix. 



Objectives for each problem were 
not clearly stated. 



Too many changes in objectives 
of group assignment. 

Suggest that assignments be 
clearer and more specific. 

Foundering because of lack 
of specificity of assignment; 
not enough time to do an 
adequate job. 



Could have used trained leaders. 



(e) Individual Study Time and Assignments: 

Positive Negativ e 

Good take home reference Poor reading facilities, 

material. 

More material than could be digested 
but good to review at home. 

Reading time is necessary and was 
adequate. 

Ample time. 

Quantity of material fine. Needed copies of some of the 

materials of the institute, i.e., 
HEW handbook and taxonomy. 

Suggest a closer relationship with 
class discussions. 
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Question 7(e) continued: 



Found some interesting 
reading. 

About the right amount. 

Good handout. 

Selection for quality recommended 
less emphasis on the volume. 



There was a s#per-abunuance of 
this. 

Ohio case problem should have 
been reviewed. 

Not enough time to learn 
information covered. 

Readings - too repetive. 



(8) Was the Institute flexible enough to allow for a change in an 
activity if the need arose? 

Yes 12 

No 0 



(9) How would you evaluate the Institute management? 



Housing 

Meals 

Institute time use 
Free time 



Very 

Satisfactory 

9 

10 



8 



Average 

3 



Very 

Unsatisfactory 



Provision for recreation 



(10) If there are any additional comments you may have regarding the 
improvement of similar institutes, please Indicate these below. 

Improve questionnaire. 

Bring family - live in dorm. 

Time to dicuss own PPBS developed. 

Shorter if possible. 
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Question 10 continued: 



Mail out more information agenda in advance. 

Use best of Maryland material; hand out samples of Minnesota and other 
PPBS data sheets; first multiple choice test was poor instrument. 

% 

More case studies; develop outline for typical vocational education 
problem. 

More coordination with similar programs conducted previously. 

More attention to time - attention to PPBS. 



APPENDIX A: PROGRAM 



OREGON INSTITUTE FOR 

PLANNING, PROGRAMMING, AND BUDGETING SYSTEMS 
August 19-30, 1968 



MAJOR OBJECTIVE 

To Enhance the General Quality of 
Planning, Programming, and Budgeting 
Systems in Vocational Education at 
State Levels 



Sponsored by 

Teaching Research Division 
Oregon State System of Higher Education 
Monmouth, Oregon 97361 



Under the auspices of a 
Grant from the United 
States Commissioner of 
Education Authorized by 
The Bureau of Research 
Under Section 4(c) of 
The Vocational Education 
Act of 1963 
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PPBS Institute Objectives 



The Oregon Planning, Programming, and Budgeting System Institute 

is focused toward state level problems in the administration of 

vocational-technical education. Specific objectives follow: 

1) To acquaint participants with educational and financial problems. 

2) Tc develop need for a systematic, information data system as a 
basis for improved decision making. 

3) To thoroughly orient participants in the structure and theory of 
Planning, Programming, Budgeting Systems. 

4) To define data streams in vocational education necessary for use in 
the PPB system. 

/ 

5) To introduce methods of analyses utilized in the PPB system. 

6) To develop a guide or model for implementation purposes in state 
divisions of vocational-technical education. 

7) To define problems of implementation and solutions as developed by 
practitioners in the field. 

8) To difine areas needing further research and development in 
application of PPBS to vocational education. 

9) To orient participants to the political processes involved in 
budgetary processes. 
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THE OREGON PPBS INSTITUTE PROGRAM 



Monday, August 19 

9:00 A.M. Welcome to Oregon College of Education 

Mr. Kersh 

Welcome to Oregon 

Mr . Loomis 

Overview of Institute 

Mr, Lee 

Research in Educational Decision Making 

Dean Goldhammer 

Coffee break and mixer 

Problems in State Level Management of Vocational- 
Technical Education 

Mr. Stanley 

Lunch 

1:30 State and Federal Finance Problems in Vocational- 

Technical Education 

Mr. Stanley 

Coffee break 

Panel Discussion on Issues Presented 

Mssrs. Stanley, Lee, Worthington, 
Swenson, Ulrich, Ringo 

4:00 Institute Administration 

Mr. McAbee 



9:30 



11:00 



12:00-1:30 



Tuesday , August 20 



8:30 A.M. 


The Need for PPBS 


at the State Level 
Mr . Cogan 


9:45 


What 


is 


PPBS? 


Mr. Neuenswander 


10:45 


What 


is 


PPBS? 


Mr . Neuenswander 
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12:00-1:30 



Lunch 



1:30 Questions and Answers on the PPBS Structure 

Mr. Neuenswander 

3:00 Clarity Philosophy, Goals, and Objectives 

Mr. Gage 

Questions and Discussion 

4:15 Institute Administration 

Mr. Mahoney 



Wednesday , August 21 



8:30 A.M. 



Group Session: Defining Criteria and Quantifying 

Objectives 

Mr. Gage 



10:30 



Reports and Discussions on the Group Sessions 

Mr. Gage 



12:00-1:30 Lunch 



1:30 



3:00 



4:15 



Developing the Program Memoranda, Terms and 
Definitions 

Mr. McAbee 

Group Session: Defining Data Needs in PPBS and 

Developing the Program Memoranda 

Mr. McAbee 

Institute Administration 

Mr. McAbee 



Thursday , August 22 

8:30 A.M. Status and Need of PPBS in Vocational Education: 
State of the Art 



9:45 

11:00 

12:00-1:30 



Mr. Legg 

Reports on Group Sessions: The Program Memoranda and 

Data Needs 

Individual Work on a PM Outline and Defining Data Needs 
for particular states 

Lunch 
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1:30 


Economic Concepts Underlying PPBS Questions and 
Answers 

Mr. Frei thaler 


3:30 


Statistical Concepts Helpful in FPBS Questions and 
Answers 

Mr. Frei thaler 


4:15 


Institute Administration 



Friday, August 23 



8:30 A.M. 


Introduction to Cost Benefit Analyses Questions and 
Answers 

Mr. Frei thaler 


10:30 


Introduction to Cost Effectiveness Analyses Questions 
and Answers 

Mr. Frei thaler 


12:00-1:30 


Lunch 


1:30 


Group Sessions: Potentials for Research and Develop- 

ment in PPBS Data Needs 


3:00 


Reports from Groups and Discussion on Potential for 
Research and Development in PPBS 

Mr. McAbee 


3:30 


Review of PPBS Structure 

Mr. McAbee 


4:30 


Institute Administration 



Monday , August 26 



8:30 A.M. 


Planning Cost Analysis of Training Programs 

Mr. McAbee 


10:30 


Group Sessions: Outline a Procedure for Cost Analysis 

of an Existing Program 


12:00-1:30 


Lunch 


1:30 


Reports and Discussion from Study Session 




3:00 


Budget Cycles, Formats, and Crosswalks 

Mr. McAbee 


4:15 


Institute Administration . 



Tuesday, August 27 

8:30 A.M. Discussion of Case Study: Cost Benefit Analysis 



10:30 


of Job Corps and Implications for State Level 
Vocational Education 

Mr. McAbee 

Group Session: Generating Alternative Programs 



12:00-1:30 Lunch 



1:30 


Reports and Discussions of Group Sessions 


3:00 


Case Study: Cost Benefit and Cost Effectiveness 

Analysis-Development of HEW Disease Control Case 


4:15 


Institute Administration 



Wednesday, August 28 

8:30 A.M. Levels and Types of Analyses Questions and Answers 



10:30 


Mr. DeLong 

Role of the Budget Examiner and Analyzing Needs 
Questions and Answers 

Mr. DeLong 



12:00-1:30 Lunch 



1:30 


Problems in Implementation: PPBS in the Washington 

State Division of Vocational Education 

Mr. Moe 


3:00 


Panel Discussion: Experiences in States Active in 
PPBS and organizing Available Resources for Imple- 
menting PPBS 

Mr. Moe. Chairman 


4:15 


Ills ti tut e Administration 
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Thursday, August 29 



8:30 A.M. 


Political Realities of the Budget Making Process 

Mr. Rothwell 


9:30 


The Legislator Looks at Educational Requests 

Senator Newbry 


10:30 


Panel: Improving Communication: A dialogue between 

Professionals, Legislators, and the Laity 

Mssrs. Rothwell, Lee, Newbry 
Hoyt, Powell 


12:00-1:30 


Lunch 


1:30 


Governor f s Office Looks at PPBS and Education Requests 

Mr. Penwell 


3:00 


Group Seasons: Planning to Implement PPBS at Home 

Mr. Lee 


5:00 


Institute Administration 



Friday, August 30 



8:30 A.M. 


Skepticism and Limitations of PPBS (Class Discussion) 

Mr. McAbee 


9:45 


Reports and Recommendations on Implementation, and 
Discussion 

Mr. Lee 


10:45 


PERTING the PPBS Task 

Mr. Lee and Selected Participants 


12:00—1:30 


Lunch 


1:00-2:00 


Institute Evaluation 

Mr. Martin 



2:00 



Departure 



Institute Speakers and Consultants 



Arnold Cogan 

State Planning Coordinator 
Office of the Governor 
State Capitol Building 
Salem, Oregon 

James V„ DeLong, Analyst 
Program Evaluation Staff 
Bureau of the Budget (Federal) 

Washington, D. C. 

William 0. Frei thaler. Professor 
Department of Economics 
University of Virginia 
Charlottesville, Virginia 

Gerald E. Gage, Associate Research Professor 
Teaching Research Division 
Oregon State System of Higher Education 
Monmouth, Oregon 

Keith Goldhammer, Dean 
School of Education 
Oregon State University 
Corvallis, Oregon 

C. Richard Hoyt, Senator 
Benton County 
Corvallis, Oregon 

Bert Y„ Kersh, Dean of Faculty 
Oregon College of Education 
Oregon State System of Higher Education 
Monmouth, Oregon 

Allen Lee, Associate Director 
Teaching Research --Division 
Oregon State System of Higher Education 
Monmouth, Oregon 

William G. Loomis 

Assistant Superintendent of Public Instruction 
Oregon State Department of Education 
Public Service Building 
Salem, Oregon 

Gary Martin, PPBS Institute Assistant 
Teaching Research Division 
Oregon State System of Higher Education 
Monmouth, Oregon 
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Harold (Hal) V. McAbee 

PPBS Institute Director 

Teaching Research Division 

Oregon State System of Higher Education 

Monmouth, Oregon 

Richard Moe, Assistant Director 
Department of Vocational, Technical and 
Adult Education 
Old State Office Building 
Olympia, Washington 

Lavor Neuenswander, Director 

Financial Management and PPBS Training Institute 
Bureau of Training, U. S. Civial Service Commission 
450 Golden Gate Avenue 
San Francisco, California 

Cleigh Penwell, Budget Director 
Department of Finance 
State Capitol Building 
Salem, Oregon 

Shirley Prather, Institute Secretary 
Teaching Research Division 
Oregon State System of Higher Education 
Monmouth, Oregon 

A1 Ringo, Director 

State Division of Vocational-Technical Education 
Oregon State Department of Education 
Salem, Oregon 

Angus Rothwell, Executive Secretary 
Coordinating Council for Higher Education 
Madison, Wisconsin 

Cecil E. Stanley, Director 
Division of Vocational Education 
Lincoln, Nebraska 

William R. Swenson, Assistant State Director 
Division of Vocational Education 
Boise, Idaho 

Walter E. Ulrich, Jr., Administrator 
Division of Vocational— Technical Education 
Salt Lake City, Utah 

Catherine Wallingford, Institute Secretary 
Teaching Research Division 
Oregon State System of Higher Education 
Monmouth , Oregon ^ 



Robert M. Worthington 
Assistant Commissioner of Education 
Division of Vocational Education 
Trenton, New Jersey 

Otto Legg, Assistant Director 

Program Planning Division 

Bureau of Vocational-Technical Education 

U. S. Office of Education 
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0. A. Brunsvold, Consultant (10) 
Area Schools and Community Colleges 
Area Schools Branch 
Department of Public Instruction 
Grimes State Office Building 
Des Moines, Iowa 50313 

Gerald D* Cresci (10) 

Assistant Chief 

Vocational-Technical Education 
Bureau of Junior College 
721 Capitol Mass, Room 250 
Sacramento, California 95814' 



Donaid M. Gilles 
Supervisor 

Trade and Industrial Education 
State Department of Education 
Salem, Oregon 97310 



Angelo G. Gillie 
Coordinator for Technical- 
Vocational Programs 
Community College System 
2327 Dole Street 
Honolulu, Hawaii 96822 

Charles Jaymes 
Program Officer 

Vocational and Technical Education 
U. S. Office of Education 
Regional Office 
760 Market Street 
San Francisco, California 94102 



Emil Kowalczyk 
Education Specialist 
Bureau of Indian Affairs 
P. 0. Box 3-8000 
Juneau, Alaska 99891 

Eraiko Kudo (Mrs • ) 

Program Analyst 

State Department of Budget and 
Finance 

Community College System 
2327 Dole Street ’ 

Honolulu, Hawaii 96822 

Otto Legg 

Assistant Director for Program 
Planning and Development 
Division of Vocational and 
Technical Education 
U. S. Office of Education 
Washington, D. C. 

William Loomis 

Assistant Superintendent of the 
Division of Community College 
State Capitol Building 
Salem, Oregon 97310 



Steve B. Marcum 

Associate Director for Program 
Operations 

Bureau of Vocational Education 
State Department of Education 
Frankfort, Kentucky 40601 



to. Morris 
1133 Sarah Lane 
Eugene, Oregon 97401 

James G. Mumper 
Program Coordination 
Department of Public Instruction 
Des Moines , Iowa 

Francis Pinkowski (10) 

Director 
Program Planning 

New Jersey Division of Vocational 
Education 

Trenton, New Jersey 



Albion Ringo 

Director of Vocational Education 
State Department of Education 
Salem, Oregon 97310 

James K. Sakai (10) 

Chief Administrator Officer 
Community College System 
2327 Dole Street 
Honolulu, Hawaii 96822 

James E. Segear 
Executive Assistant 
Office of Assistant Commissioner 
New Jersey Division of Vocational 
Education 

Trenton, New Jersey 08625 

Samson Shigetomi, Provost 
Community College System 
2327 Dole Street 
Honolulu, Hawaii 96822 



Cecil E. Stanley 
Assistant Commissioner for 
Vocational Education 
Tenth Floor 
Vocational Education 
State House 
Lincoln, Nebraska ' 

Glen H. Strain (10) 
Divisional Coordinator 
Tenth Floor 
Vocational Education 
State House 
Lincoln, Nebraska 

Burton A. Talmage 
Treasurer 

Utah Technical College at 
Salt Lake 

4600 South Redwood Road 
Salt Lake City, Utah 84107 

Walter E. Ulrich, Jr. 
Administrator 

Vocational-Technical Education 
1400 University Club 
136 East South Temple 
Salt Lake City, Utah 

E. P. Hilton 

State Department of Education 
F rankf ur t , Kentucky 



John Radvany 
Director 

Center for Occupational Experimentation 
and Demonstration 

Newark Man Power Training Skills Center 
87 N. Broadway 
Newark, New Jersey 
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Capar, Wesley, "Planning-Programming-Budget System Terms and 
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Memo, February 8, 1967. 
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"Selected Cases On PPBS," 
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8) The Rolling Stock Auto Company 
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September 20, 1966, Office of Economic Opportunity. 

Terrey, John N., "Program Budgeting And Other Newer Management Tools 
In Higher Education: A Description- And Annotated Bibliography," 
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Kaufman, Roger A., et.al., "Methods-Means Analysis In Education." 
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APPENDIX D 



OREGON PPBS INSTITUTE 
August 19-30, 1968 



August 18: 



August 19: 



August 20: 



August 21: 



August 22: 



. Reading Assignments 

Please complete the pre-test and hand in Monday 
morning . 

Examine the program. 

Other materials in your kit. 

Planning-Programming-Budgeting Hearing; U. S. 

Congress Administrative Bulletin. 

What is a PPB System? 

Available References, Oregon PPBS Institute. 

The Planning-Programming-Budgeting System: Progress 

and Potential; pp 1-10, 46-48. 

The Concept of a Program Budget. 

Bulletin 68-9, Executive Office of the President, 

BOB, pp 1-11, PPBS Budgeting System. 

Progress and Potentials; pp 86-127 

What's Ahead for Vocational Education, No. 4, USOE. 

Complete the readings in PPBS: Progress and Potentials. 

Program Budgeting and Other Newer Management Tools 
in Higher Education. 

Cost Effectiveness: An Introduction and Overview. 

Using Cost-Benefit Analysis in Planning and 
Evaluating Vocational Education. 

Supplemental: Measuring Benefits of Government 

Investments . 

Office of Economic Opportunity Paper on Evaluation. 



August 23: 



August 23: 


Review Assignments for August 22. 

Concepts and Philosophy of a PPB System. 
Bulletin No. 68-9 - Attachment B. 

Cost Analysis for PPB Cost-Benefit Studies. 


August 26: 


Benefit-Cost Estimates for Job Corps. 

PPBS: Progress and Potentials, pp, 182-191. 


August 27: 


Systems Analysis Techniques for PPB. 
Disease Control Programs. 

What's Ahead For Vocational Education? 


August 28: 


The Politics of the Budgetary Process.. 

PPBS: Progress and Potentials, pp, 191-225. 


August 29: 


PPBS: Progress and Potentials, pp, 225-235. 

PPBS: Rationale, Language, and Idea Relationships 
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APPENDIX E 



CRITERIA FOR FEASIBILITY OF OCCUPATIONAL PROGRAMS 



Introduction , These Criteria represent premises or standards on 
which judgments concerning the feasibility of continuing or implemfenting 
occupational curricula may be based, 

1) O bjective s. The objectives of the occupational curriculum 
under consideration will be specified. The sources for the objectives 
should be given. Consistency of the curriculum objectives with ldng 
range goals and philosophy of the institution should be shown. 

2) Manpo wer Requirement Needs . The need for trained manpower- 
resulting from the occupational training program must be shown. 

Definite manpower needs must be shown at either the local, state, 
regional, or national levels. This listing also constitutes an order 
of priority.* 

3) Pr ogram Costs . These shall be specified in terms of cost 
per traineeT and recognized as being reasonable. Data shall include 
both financial (out-of-pocket) and real resource (total economic) costs. 

A cost effectiveness study will be presented. Cost benefit studies 
will be made if possible. 

4) Financing . Adequate and realistic funding shall be available. 
Both actual and potential sources of financing should be reported. 

These include local, state, and federal sources of funding. Local 
sources include tax income, fees, and tuition, income from other 
private sources should also be included when appropriate. 

5) Student Availability . Data showing student interest and 
availability on a would enroll basis shall be given. These data should 
be listed on a local, state, and regional basis. Evidence of potential 
students’ capabilities of completing the program should be assessed. 

6) Gover ning Board and Advisory Committee Approval . New programs 
should have approval of the local governing board (s) and advisory 
committee(s) , and the Oregon State Advisory Council for Vocational 
Education. If expenditures of state or federal funds are involved, new 
programs must have approval from the Advisory Council and the State Board 

of Education . 

7) The Occupational Educ ation Curriculum . A curriculum outline 

• shall be furnished. It should be brief but detailed enough to show the 
following; 

- a) Kow the curriculum will achieve the stated objectives. 

b) Course outlines showing organization for recycling the 
curriculum. 
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c) Congruency with Dictionary of Occupational Titles. 

d) Various entry and exit levels. 

e) Plans to fill class vacancies as they occur. 

f) Suitability for the target population. 

g) Organization for individually paced instruction. 

h) Plan for evaluation of trainee progress. 

i) Plan for evaluation of the curriculum. 

j) Availability of related educational experiences. 

8) Co nsideration of Alternatives . These considerations should 
include : 

a) Other curricula which could conceivably meet the stated 
ob j ectives . 

b) Other curricula which could meet different objectives. 

c) Other institutional basis for the program. 

d) Effect of not offering the program. 

9) Instructor Availability . A qualified teacher should be 
available” If notT assess the potential for recruiting the needed 
instructional assistance. 

10) Potential for Development . This criterion should be 
examined on the basis of short and long term needs. Effect of long 
term, growth potential should be examined with respect to manpower 
requirements, costs, student availability, available financing, and 
curriculum expansion. 



APPENDIX F 



A BENEFIT-COST PROBLEM 

by Dr. Joseph McGidney and Mr. William Nelson 
Ohio State Universtiy 



I. The Situation 

The construction of a new educational facility is being considered 
by the board of education and the local school administrators in conjunc- 
tion with the state education department. As a member of state education 
department, you are asked to develop a benefit-cost study comparing the 
expected benefits and costs of constructing a two-year vocational school 
versus an academic junior college. 

The local school system has three vocational school and seven 
junior colleges at the present time. Data given in Tables 1 and 2 have 
been derived from the best estimates of future construction and operating 
costs and from historical data on employment and salary levels of 
graduates from the present school system. The cost figures for the 
junior college are based on an annual graduating class of one hundred. 
Costs of the vocational school are based on an annual graduating class 
of fifty. 

Discount rates and present value of future earnings are given in 
Tables 3 and 4. 



II. The Assumptions 

1) Costs given in Tables 1 and 2 are the summation of all costs 
of the two-year program for a graduating class. Initial 
building and equipment costs have been amortized to an annual 
basis. 

2) Benefits given in Tables 1 and 2 are assumed to be the average 
annual earnings per graduate. Initially, assume the effect 

of education has a ten year life. 

3) Annual salaries of nongraduates were derived from the earnings 
of high school graduates. 



III. The Analysis 

Step A. What are the benefit-cost ratios and the optimum economic 
decision using: 

1) An interest rate of 0.00 



- 90 - 



2 ) 



An interest rate of 0.05 
3) An interest rate of 0.10 

Step B. Would the results he different if net present value were 
used as the criteria for evaluation? Why? 



Step C. What, if any, are the critical factors in your analysis? 
What would be the effect of changing those factors? 



Step D. What further information would be necessary to make a 
definite recommendation in a real situation? 
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IV. Discussion of Analysis 



Steps A and B. 







Junior 


Vocational 




College 


School 


Benefit-Cost Ratios 






0.00 


2.400 


2.100 


0.05 


1.712 


1.629 


0.10 


1.262 


1.306 


Net Present Value 






0.00 


$1,4000,000 


$550,000 


0.05 


712,000 


314,500 


0.10 


261,500 


152,8000 



Step C. 







Critical Factors 


effect 



1) Annual Salaries 

2) Future Costs 

3) Interest Rates 

4) Size and Number of Schools 

5) Cause-Effect Relationship between 
Education and Income 

6) Direction of Economic Growth in 
the Area 

7) Noneducational Investment Alternatives 



Step C. (continued) 





Critical Factors 


Effect 


8) 


Social, Cultural, and political 
Aspects 




9) 


Ability and Interest of Students 




10) 


Migration of Graduates 




11) 


Local, State or National Analysis 




12) 


Effect of Graduates Upon the 
Wage Levels 




13) 


Specific Occupational and 
Academic Programs in Schools 




14) 


Future Level of Unemployment 




- — ■ ■ ■ . 1 * " 



Step D. Additional Information 

1) specific Location of Schools 

2) Direction of Economic Growth 

3) Manpower Projections (Labor Market) 

4) Student Population Projections 

5) 

6 ) 

General Comments 

1) Net present value and benefit-cost ratios do not necessarily lead 
to the same conclusion. 

2) Changing the interest rate can change the relative position of 
investment alternatives • 
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3) There are a very large number of implicit assumptions in any 
benefit-cost analysis. 

4) Benefit-cost analysis is only one of many criteria for program 
evaluation. 
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APPENDIX G 



PROGRAM EVALUATION CHECKLIST 



The following list of questions has been assembled for the use 
both of consumers and producers of key documents in the Planning— 
Programming-Budgeting System. It focuses mainly on the analyses to be 
reflected in the Program Memoranda (PM) although some points relate to 
the Program and Financial Plans (PFP). 

I. OBJECTIVES 

Perhaps the most difficult and important task of analysis of 
programs in the public sector is being clear on objectives and the 
contribution of specific programs to specific objectives. 

1) What assertions are made about overall national objectives? 
What authority is quoted? The President? The Congress? The Agency 
Head? How specific and concrete is the statement of objectives? To 
what extent is the Government committed to these objectives? What form 
does the commitment take? Presidential statements? Statutes? Agree- 
ment with Congressional committees? Moral obligations? How much 

room for interpretation is there? For example, is it clear over what 
time period the objectives should be met? Does the PM translate 
general, abstract, vague objectives into operationally meaningful ones? 

2) Is It clear what should be the Federal Government’s respons- 
ibility for the cited or proposed objectives? Has the alternative of 
greater contribution by State and local agencies or the private sec 
been considered? Should it be a Federal responsiblility at all? 

3) Is the relationship between broad national goals and specific 
agency goals clearly stated? Is it reasonable? What policies, 
legislation and statements of fact are implicit in the objectives? 
Should they be made explicit? Are they right? 

4) Are the agency goals stated in terms which have operational 
meaning? Have they been expressed quantitatively? Is there a time 
table for accomplishment? 

5) Has the relationship among related goals in different parts 
of the same agency or in different agencies been made explicit (e.g., 
the War on Poverty)? Does the analysis adequately take into account 
other efforts to accomplish these goals — efforts by other agencies of 
the Federal Government, by State or local governments, or by the 
private sector? Is the impact of the agency's programs on the goals of 
other agencies noted? 

6) Have the agency's priorities among objectives and programs 
been made clear? If more, or fewer, funds are available than in the 
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basic case assumed by the agency, is it evident what the agency would 
propose to add or delete from its programs? 

7) What is the evidence that the alleged "need" for goods and 
services exists in the sense that people would be willing to pay for 
the service? Given this evidence, does the analysis show that the 
agency's proposed scale of operation is best? Or has the agency 
proposed fulfillment of all potential "need" for the service by 
employing an implicit price level of zero? On the other hand, has it 
stopped short of fulfilling demand willing to meet the proper price? 
Why? Was a budget constraint the reason for this? Can a better 
budget justification be made if this is the case (perhaps as the result 
of further study)? 



II. MEASURES OF EFFECTIVENESS 

The private sector (and certain business-like governmental 
activities) have profits which serve as a guide to performance; most 
governmental agencies have to develop their own specific indicators of 
performance. This is often difficult to do well and errors of various 
kinds are frequent. 

1) To what extent are the tangible, measurable outputs of the 
program used by the agency adequate indicators of the extent to which 
the overall purposes of the program are being achieved? If inadequate, 
what better measures of performance might be used? 

2) Are there clearly identified criteria offered by which to 
judge program performance? How adequate do they seem to be? Are there 
some alternative criteria that should be considered? 

3) Have outputs been quantified to the extent useful? Where 
quantitative measures of program performance are missing are there 
logical, convincing qualitative assessments? Has the agency gone too 
far and attempted to quantify the unquant if iable? Is it clear where 
dependence on the quantitative outputs leaves off and judgment takes 
over? 



4) Are there close substitutes for these outputs which will 
affect the demand for them? Are these substitutes considered in the 
analysis? 



III. FINANCIAL DATA 

Whereas it is often very difficult to define and obtain good 
measures of program effectiveness, the cost side is usually easier to 
cope with. And often a better understanding of costs can be very useful 
in improving program performance. 
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1) Does the PM present the costs that are relevant to the issues 
analyzed? Have systems costs estimates been developed and presented? 

How complete and how accurate are these estimates? Have any valuable 
inputs been ignored? Are all directly related support, training and 
construction costs included? What additional cost analytic efforts 
are suggested by the quality of the agency* s cost data? 

2) Does the analysis describe the bases for the cost estimates? 

Do the estimates look reasonable? 

3) Have sunk cost.o — sacrifices — been omitted as they should be, 
except where there are significant salvage values? Have they been 
considered where they should be i.e. in the evaluation of the past 
program? 

4) In systems with multiple products, have only the nonavoidable 
costs been charged to a given program, or have common costs been 
arbitrarily allocated? What is the purpose of the allocations? Have 
popular programs been saddled with part of the burden of less attractive 
programs? That is, have costs been shifted from where they should 
properly be charged to where they can most easily be funded? Have costs 
been shifted from products with easily measured output to products with 
only qualitative outputs? For example, in a multipurpose dam project, 
are costs overailocated to recreation and underallocated to irrigation 
or power products? 

5) Is the agency too optimistic in its assumptions on funds 
likely to be available for its programs? What effect should more modest 
assumptions have on the conclusions? 

6) Is there available a record of successive agency cost estimates 
over time of new programs? What does this suggest about the accuracy of 
current claims? Why shouldn*t the agency be asked to have available a 
record of its estimates? 

7) Is there a sensitivity analysis to show the effect of uncer- 
tainty in cost estimates on conclusions? What evidence is adduced in 
support of its cost estimates? 

8) Have future benefits and costs been discounted? For example, 
if a proposed project is compared with existing ones for which future 
savings are expected, are those future savings discounted? On the other 
hand, is the case for discounting future outlays and benefits negated by 
the existence of a brief time span for the project, or by uncertainties? 

9) Is the chosen discount rate reasonable, i.e., does it reflect 
alternative uses of investible funds? How sensitive are conclusions on 
preferred programs to the choice of discount rates? 
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IV. SOME ADDITIONAL POINTS 



Treatment of Alternatives 

1) How thoroughly have alternative methods for accomplishing the 
same objectives been analyzed? Are there alternatives involving 
different tactics or techniques, different modes of production (more 
or less capital intensive), different priorities, different mixes of 
Federal vs. State and local expenditures, different financing methods, 
different legislative requirements? Were different levels for* the 
agency* s preferred program studied? Are mixtures of systems considered 
among the alternatives? 

2) Have the costs and effectiveness of the major alternatives 
been properly assessed? Have tradeoffs been made between present and 
proposed programs? Were the criteria, costing methods and time span 
the same as for the proposed program, i.e., was the comparison fair? 

3) Has adequate account been taken of the need to examine 
programs at the margin? Quite often the issue is not whether to have 
a program at all, but whether to make marginal changes in the program 
level or composition. Does the analysis deal with specific issues in 
terms of the marginal benefits and marginal costs? Does the data 
displayed show the effect of marginal changes or current marginal 
operations? 

4) Has the presentation of program data in terms of arithmetic 
averages concealed important information about the spread in values? 

If an analysis supports certain conclusions about a program based on 
these averages, might there nevertheless be important exceptions? What 
are they? Should they be examined? 

5) Are the alternatives analyzed real options for the agency 

head? Does he really have a choice? On the other hand, is there inability 
or unwillingness to consider radically different programs? 

6) Have the constraints on the program area been explicitly 
treated? What are they? Have ways of modifying the constraints been 
explored? What reasons have been given for rejecting alternative 
objectives or programs? Efficiency? Statutory prohibitions? Organ- 
izational limitations? Lack of trained personnel? Lack of capital? 
Inadequate technology? 



Models 

1) Is there presented a model or simplified, quantitative 
description of some important aspects of the program? If this hasn*t 
been attempted for a broad program area, has it been attempted for some 
important parts? Has the analyst failed to structure the important 
relationships explicitly? How adequately does the model seem to explain 
observed phenomena? Does it rest on structural relationships or is it 
based entirely on statistical correlation, i.e., is a theory offered? 
Does it seem palusible? 
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What 



2) Can you suggest specific improvements on the model? 
factors should be included that have been left out? 

3) Has the analyst engaged in excessive modelism, i*e. , does his 
work reflect more interest in the model than in the real world? Has he 
focused on manipulating a computational method at the expense of dealing 
with important factors in a sensible way? 

4) Are there severe suboptimizations? Have elements that should 

be regarded as variables by the agency head or the President been allowed 
to vary, or are there restrictions imposed in the process of dividing 
the analysis into lower level problems? 



T reatmen t of the Future 

1) Has the proper span of years been considered? Does the PFP 
or the PM go too far, or not far enough, into the future? Beyond the 
imm ediate period, is there simply a mechanical projection of recent 
trends which fails to fake adequate account of possible changes in 
policy, technology, costs, or availability of funds? 

2) Has an explicit, time phased, decision strategy been presented 
which allows for options being opened up and foreclosed, research 
being done, experiments being carried out, and data gathered? Alter- 
natively, is the program area characterized by a collection of unstruc- 
tured ad hoc programs and decisions without a coherent theory? If the 
latter, what should be done to improve the situation? 

3) Does the PM show a range of possible future options as to 
program objectives and means of achieving these objectives? Does it 
show over what period. of time in the future it should become possible to 
choose among these options? Does it show what evidence should dictate 

a choice among options? 

4) Is the proposed program level shown dependent on how well the 
program performs over the next year or two? Is there any discussion of 
the evidence that should dictate cutting off the program entirely or 
redirecting it drastically? 

5) Have economic, demographic, cultural trends been explicitly 
taken into account? How adequately? 



Uncertainty 

Program uncertainty can be of various kinds: e.g., there can be 

uncertainity because of a lack of clarity in objectives, risk (statistical 
uncertainty), new technology, factors external to the program (e.g., 
the future level of economic activity), inadequate bases for estimating 
costs, and inadequate bases for estimating consumer demand. 
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1) Have the adequacies and inadequacies of data on major issues 
been discussed? Where data are inadequate, does the ?M state what 
action will be taken to improve the situation? 

2) Aside from data problems, are there important underlying 
uncertainties about program effectiveness or costs? Is there important 
technological uncertainty? Does the PM state what additional analysis 
or research is being undertaken to reduce these uncertainties? 

3) Have sensitivity tests been carried out to determine how the 
uncertainties affect program recommendations? (e.g.. Has a "break, 
even" analysis been arade which shows what the value of an uncertain 
parameter would have to be in order to get equivalence among- alternative 

programs? 



V. GENERAL 

1) Have the programs covered in the PM been described clearly, 
concisely, and quantitatively? 

2) Are critical assumptions, relationships and facts spelled out? 
In any program area, a few factors usually are much more important than 
the rest. Does the analysis identify and focus on these critical 
factors? Or does it handle the important ones by assumption and 
concentrate on less important ones? 

3) Have the most important data on a program area been presented 
or are key assertions made without supporting data? Should the agency 
be asked to prepare and forward additional data prior to the Fall budget 

review? 

4) lias the program "base" been analyzed or has the analysis been 
limited largely or entirely to new programs? 

5) Have the recommendations presented on a program area been 
compared with earlier agency recommendations on the same subject? Have 
data on actual program performance been compared with earlier agency 
claims for the program? 

6) Is the recommended program composition and level based on an 
overall analysis of (a) need or effective demand; (b) an arbitrary rule 
of thumb (e.g. x percent more than last year's level): (c) Presidential 
commitment; (d) an estimate of the level that the Congress will accept; 
(e) other considerations? 

7) Where the analysis is inadequate or incomplete, does the PM 
indicate what future work is planned to improve it; the date to be 
gathered, criteria to be applied, method of analysis to be used. 
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APPENDIX K 



OREGON PPBS INSTITUTE 
Inventory Instrument 

Section I 

Instructions : This instrument is designed to assess your present 

knowledge of the PPB system. In each group of three items you are to 
rank the items in the terms of most correct (1) to least correct (3) . 

For example, in the following a one has been placed by the first item 
as it is the most correct of the three statements. The second item is 
least correct, thus a three is placed by it. The last item would be 
given a two. 

Example : (1) The "Scarlet Letter" was written by Nathaniel Hawthorne. 

(3) Thomas Jefferson was the fifth President of the United States. 
(2) The Southwestern Section of the United States is a nice 
place to live. 



1. ( ) Reorganization of state level vocational education adminis- 

tration is not needed. 

( ) PPB in operation, tends to decentralize decision making. 

( ) Most vocational education programs are administered under the 
umbrella of a single agency in most states. 

2. ( ) One of the important aspects of PPBS is cost projection into 

future years. 

( ) The only real financial problem in vocational education is 
lack of sufficient funds. 

( ) PFB3 is another way of presenting the traditional budget 
requests and is not worth the extra effort involved. 

3. ( ) Several well organized information systems are tested and 

available to state level vocational education decision makers. 

( ) Planning, Programming, and Budgeting Systems (PPBS) is a 
statistically based, systems, and computer approach which 
takes human error out of decision making. 

( ) The PPB system will provide definite information relative to 
available alternatives for adequate decision making. 

4. ( ) There are a few problems concerning allocation of scarce monies 

to a variety of vocational education needs. 

( ) Implementing new systems of budgeting (PPBS hopefully) will 
engender many of the same problems as other changes. 

( ) Program categories should coincide with classification of 
budget inputs. 

5. ( ) The most important reason for utilizing PPBS is to maximize 

program benefits for the dollar expended. 

( ) PPBS is basically a method to save money. 

( ) A statement of objectives in the PPB system should be made in 
behavioral terras. 
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6. ( ) PPBS means Politics, Priorities, and Budget Systems. 

( ) There are few materials and resource persons to assist 
educators willing to implement PPBS. 

( ) There are sufficient data existing which indicate clearly 
which vocational programs are the most valuable. 

7. ( ) It is satisfactory to interchange philosophy, goals, and 

objectives in application of the PPB system of budget 

information. . 

( ) The best administrative arrangement for PPBS specialists is 

staff relationship to the budget director. 

( ) PPBS, as applied to education endeavors, needs much research 
and development for full realization of its potential. 

g # ( ) it is not necessary to use cost utility analyses in PPBS. 

( ) Education is a human affair and the systems analysis people 
should stay out of it. 

( ) PPBS stimulated program development with its emphasis on 
alternatives . 

9. ( ) There are well defined guides available for implementing PPBS 

at state and local levels. 

( ) There is still much room for value judgments in application 
of PPB systems. 

( ) Analyses in PPBS should begin with the resources assigned to 
a program. 

10. ( ) Program benefits and costs cannot be separated in analyses in 

PPBS. . , . 

( ) PPBS is such a logical system that the decisions can be made 

without reference to politics. 

( ) A decision maker is only interested in the program which is 
most economical per unit of output. 



Section II 

Please respond to the following questions: 

1. What value does PPBS hold for vocational education? Why? 
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2 , List two of the most important aspects about the PPB system as you 
now understand it. 

( 1 ) 



( 2 ) 
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PPBS INSTITUTE QUESTIONNAIRE 



(1) Was the Institute staff responsive to the needs and problems of 
the participants? (Give one positive and one negative example) 

Positive: 



Negative: 



(2) What activities were most productive for you and which experiences 
were least productive, e.g., large presentations, small training 
group work, panel work, individual study work, etc. 

Host productive — 



Least productive 



(3) In this Institute, we were concerned with giving information 

providing for the planning of implementation for PPBS , and for 
development of a set of guidelines for applying such a PPB system 
in your state. Would you have preferred more or less experience 
in each of these areas? 

(a) Giving information: More Less 

Comments : — 



(b) Planning for implementation of PPBS: 



More Less 



Comments : 



(c) Development of guidelines for applying PPBS: 



More 



Less 



Comments : 



(4) Generally speaking, were our instructional objectives relevant to 
your needs in the field? 

Yes No 

Most relevant: — 

Least relevant: __ — — 

(5) What instructional objectives should be added to make the Institute 
more relevant? 

Comments : — — — 



(6) What effects do you think this Institute will have on the 
performance of your job next year? 

Comments : — 



(7) Please give a positive and negative comment on each of the 
following Institute activities. 

(a) Large group presentations : 

Positive Comment: . - — 



Negative Comment: 



(b) Training group activities : 



Positive Comment: 



Negative Comment: 



(c) Panel discussions : 
Positive Comment: 



Negative Comment: 



(d) Small group activitie s : 
Positive Comment: 



Negative Comment : 



(e) Individual study time and assignments : 
Positive Comment: 



Negative Comment: 



(8) Was the Institute flexible enough to allow for a change in an 
activity if the need arose? 

Comments: . — 



(9) Kow would you evaluate the Institute management? 

Very Very 

Satisfactory Unsatisfactory 

(a) Housing . — — - — — 

(b) Meals — 

(c) Institute time use _____ 
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(d) Free time for informal 
visiting and exchange 
of ideas 

(e) Provision for recrea- 
tional activity 



(10) If there are any additional comments you may have regarding the 
improvement of similar institutes, please indicate these below. 
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